Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.2,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1300,
 'ffn_num_layers': 1,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.566940
Epoch 1
Validation binary_cross_entropy = 0.454473
Epoch 2
Validation binary_cross_entropy = 0.635275
Epoch 3
Validation binary_cross_entropy = 0.763915
Epoch 4
Loss = 1.5881e+00, PNorm = 54.1594, GNorm = 15.1880, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.666729
Epoch 5
Validation binary_cross_entropy = 0.598705
Epoch 6
Validation binary_cross_entropy = 0.551113
Epoch 7
Validation binary_cross_entropy = 0.584737
Epoch 8
Validation binary_cross_entropy = 0.501742
Epoch 9
Loss = 1.2938e+00, PNorm = 54.3336, GNorm = 23.2423, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.642838
Epoch 10
Validation binary_cross_entropy = 1.202008
Epoch 11
Validation binary_cross_entropy = 0.731559
Epoch 12
Validation binary_cross_entropy = 0.556026
Epoch 13
Validation binary_cross_entropy = 0.522331
Epoch 14
Loss = 1.2801e+00, PNorm = 54.4920, GNorm = 13.1335, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.529742
Epoch 15
Validation binary_cross_entropy = 0.592403
Epoch 16
Validation binary_cross_entropy = 0.667754
Epoch 17
Validation binary_cross_entropy = 0.641801
Epoch 18
Validation binary_cross_entropy = 0.598228
Epoch 19
Loss = 8.0177e-01, PNorm = 54.6203, GNorm = 19.1464, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.459243
Epoch 20
Validation binary_cross_entropy = 0.408952
Epoch 21
Validation binary_cross_entropy = 0.447989
Epoch 22
Validation binary_cross_entropy = 0.520740
Epoch 23
Validation binary_cross_entropy = 0.479645
Epoch 24
Loss = 9.1240e-01, PNorm = 54.7456, GNorm = 17.4589, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.441281
Epoch 25
Validation binary_cross_entropy = 0.390177
Epoch 26
Validation binary_cross_entropy = 0.508627
Epoch 27
Validation binary_cross_entropy = 0.588326
Epoch 28
Validation binary_cross_entropy = 0.419600
Epoch 29
Loss = 1.1265e+00, PNorm = 54.8852, GNorm = 13.9875, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.346600
Epoch 30
Validation binary_cross_entropy = 0.406522
Epoch 31
Validation binary_cross_entropy = 0.663103
Epoch 32
Validation binary_cross_entropy = 0.482193
Epoch 33
Validation binary_cross_entropy = 0.365106
Epoch 34
Loss = 1.0475e+00, PNorm = 55.0329, GNorm = 44.9606, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.353205
Epoch 35
Validation binary_cross_entropy = 0.486970
Epoch 36
Validation binary_cross_entropy = 0.589245
Epoch 37
Validation binary_cross_entropy = 0.372652
Epoch 38
Validation binary_cross_entropy = 0.346223
Epoch 39
Loss = 1.1574e+00, PNorm = 55.2009, GNorm = 24.6834, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.380106
Epoch 40
Validation binary_cross_entropy = 0.517804
Epoch 41
Validation binary_cross_entropy = 0.462898
Epoch 42
Validation binary_cross_entropy = 0.356341
Epoch 43
Validation binary_cross_entropy = 0.346255
Epoch 44
Loss = 7.0878e-01, PNorm = 55.3699, GNorm = 31.9379, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.372075
Epoch 45
Validation binary_cross_entropy = 0.560477
Epoch 46
Validation binary_cross_entropy = 0.435370
Epoch 47
Validation binary_cross_entropy = 0.345455
Epoch 48
Validation binary_cross_entropy = 0.370210
Epoch 49
Loss = 6.9777e-01, PNorm = 55.5589, GNorm = 10.9998, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.561927
Epoch 50
Validation binary_cross_entropy = 0.409820
Epoch 51
Validation binary_cross_entropy = 0.348705
Epoch 52
Validation binary_cross_entropy = 0.366047
Epoch 53
Validation binary_cross_entropy = 0.388577
Epoch 54
Loss = 6.5626e-01, PNorm = 55.7850, GNorm = 8.4312, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.339055
Epoch 55
Validation binary_cross_entropy = 0.343887
Epoch 56
Validation binary_cross_entropy = 0.460622
Epoch 57
Validation binary_cross_entropy = 0.344446
Epoch 58
Validation binary_cross_entropy = 0.351684
Epoch 59
Loss = 5.2801e-01, PNorm = 56.0153, GNorm = 8.6068, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.384837
Epoch 60
Validation binary_cross_entropy = 0.408457
Epoch 61
Validation binary_cross_entropy = 0.413719
Epoch 62
Validation binary_cross_entropy = 0.420047
Epoch 63
Validation binary_cross_entropy = 0.451413
Epoch 64
Loss = 3.7504e-01, PNorm = 56.2867, GNorm = 7.8854, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.491410
Epoch 65
Validation binary_cross_entropy = 0.417680
Epoch 66
Validation binary_cross_entropy = 0.460339
Epoch 67
Validation binary_cross_entropy = 0.436354
Epoch 68
Validation binary_cross_entropy = 0.921237
Epoch 69
Loss = 4.1767e-01, PNorm = 56.6210, GNorm = 14.4419, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.470940
Epoch 70
Validation binary_cross_entropy = 0.428297
Epoch 71
Validation binary_cross_entropy = 0.546893
Epoch 72
Validation binary_cross_entropy = 0.508878
Epoch 73
Validation binary_cross_entropy = 0.468986
Epoch 74
Loss = 3.2976e-01, PNorm = 56.9534, GNorm = 6.0186, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.549281
Epoch 75
Validation binary_cross_entropy = 0.416754
Epoch 76
Validation binary_cross_entropy = 0.429977
Epoch 77
Validation binary_cross_entropy = 0.510794
Epoch 78
Validation binary_cross_entropy = 0.429023
Epoch 79
Loss = 3.4638e-01, PNorm = 57.2612, GNorm = 7.8938, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.430433
Epoch 80
Validation binary_cross_entropy = 0.428682
Epoch 81
Validation binary_cross_entropy = 0.587179
Epoch 82
Validation binary_cross_entropy = 0.509848
Epoch 83
Validation binary_cross_entropy = 0.467180
Epoch 84
Loss = 4.9844e-01, PNorm = 57.5656, GNorm = 27.0798, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.466036
Epoch 85
Validation binary_cross_entropy = 0.439835
Epoch 86
Validation binary_cross_entropy = 0.506193
Epoch 87
Validation binary_cross_entropy = 0.485202
Epoch 88
Validation binary_cross_entropy = 0.443114
Epoch 89
Loss = 3.0218e-01, PNorm = 57.8432, GNorm = 6.0302, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.461642
Epoch 90
Validation binary_cross_entropy = 0.459558
Epoch 91
Validation binary_cross_entropy = 0.568475
Epoch 92
Validation binary_cross_entropy = 0.503231
Epoch 93
Validation binary_cross_entropy = 0.485223
Epoch 94
Loss = 1.9707e-01, PNorm = 58.1078, GNorm = 6.0169, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.528715
Epoch 95
Validation binary_cross_entropy = 0.489477
Epoch 96
Validation binary_cross_entropy = 0.457263
Epoch 97
Validation binary_cross_entropy = 0.475016
Epoch 98
Validation binary_cross_entropy = 0.479287
Epoch 99
Loss = 1.1707e-01, PNorm = 58.3585, GNorm = 4.1456, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.688951
Epoch 100
Validation binary_cross_entropy = 0.534508
Epoch 101
Validation binary_cross_entropy = 0.525218
Epoch 102
Validation binary_cross_entropy = 0.524949
Epoch 103
Validation binary_cross_entropy = 0.536288
Epoch 104
Loss = 1.2186e-01, PNorm = 58.6047, GNorm = 4.8667, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.575633
Epoch 105
Validation binary_cross_entropy = 0.538753
Epoch 106
Validation binary_cross_entropy = 0.552198
Epoch 107
Validation binary_cross_entropy = 0.534566
Epoch 108
Validation binary_cross_entropy = 0.538771
Epoch 109
Loss = 3.7285e-01, PNorm = 58.8385, GNorm = 9.9448, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.596825
Epoch 110
Validation binary_cross_entropy = 0.586180
Epoch 111
Validation binary_cross_entropy = 0.559488
Epoch 112
Validation binary_cross_entropy = 0.624339
Epoch 113
Validation binary_cross_entropy = 0.550765
Epoch 114
Loss = 2.5946e-01, PNorm = 59.0864, GNorm = 4.7888, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.528324
Epoch 115
Validation binary_cross_entropy = 0.557926
Epoch 116
Validation binary_cross_entropy = 0.566257
Epoch 117
Validation binary_cross_entropy = 0.595873
Epoch 118
Validation binary_cross_entropy = 0.607139
Epoch 119
Loss = 9.5711e-02, PNorm = 59.3293, GNorm = 4.1636, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.641676
Epoch 120
Validation binary_cross_entropy = 0.694006
Epoch 121
Validation binary_cross_entropy = 0.749595
Epoch 122
Validation binary_cross_entropy = 0.788820
Epoch 123
Validation binary_cross_entropy = 0.773571
Epoch 124
Loss = 1.8088e-01, PNorm = 59.5348, GNorm = 9.6966, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.730285
Epoch 125
Validation binary_cross_entropy = 0.702240
Epoch 126
Validation binary_cross_entropy = 0.634943
Epoch 127
Validation binary_cross_entropy = 0.580177
Epoch 128
Validation binary_cross_entropy = 0.623759
Epoch 129
Loss = 1.3387e-01, PNorm = 59.7085, GNorm = 3.1801, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.589861
Epoch 130
Validation binary_cross_entropy = 0.584822
Epoch 131
Validation binary_cross_entropy = 0.608784
Epoch 132
Validation binary_cross_entropy = 0.639800
Epoch 133
Validation binary_cross_entropy = 0.655309
Epoch 134
Loss = 1.3746e-01, PNorm = 59.8851, GNorm = 9.0017, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.669777
Epoch 135
Validation binary_cross_entropy = 0.687149
Epoch 136
Validation binary_cross_entropy = 0.667912
Epoch 137
Validation binary_cross_entropy = 0.638384
Epoch 138
Validation binary_cross_entropy = 0.672293
Epoch 139
Loss = 1.0222e-01, PNorm = 60.0715, GNorm = 1.9456, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.685760
Epoch 140
Validation binary_cross_entropy = 0.705304
Epoch 141
Validation binary_cross_entropy = 0.706542
Epoch 142
Validation binary_cross_entropy = 0.717232
Epoch 143
Validation binary_cross_entropy = 0.707539
Epoch 144
Loss = 4.6457e-02, PNorm = 60.2493, GNorm = 2.1861, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.713966
Epoch 145
Validation binary_cross_entropy = 0.718743
Epoch 146
Validation binary_cross_entropy = 0.732534
Epoch 147
Validation binary_cross_entropy = 0.784680
Epoch 148
Validation binary_cross_entropy = 0.803028
Epoch 149
Loss = 2.4386e-02, PNorm = 60.3897, GNorm = 1.4057, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.795728
Epoch 150
Validation binary_cross_entropy = 0.805280
Epoch 151
Validation binary_cross_entropy = 0.817259
Epoch 152
Validation binary_cross_entropy = 0.822119
Epoch 153
Validation binary_cross_entropy = 0.906081
Epoch 154
Loss = 9.7425e-02, PNorm = 60.4944, GNorm = 15.5301, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.894630
Epoch 155
Validation binary_cross_entropy = 0.844739
Epoch 156
Validation binary_cross_entropy = 0.932509
Epoch 157
Validation binary_cross_entropy = 0.832527
Epoch 158
Validation binary_cross_entropy = 0.828516
Epoch 159
Loss = 4.8862e-02, PNorm = 60.6290, GNorm = 0.6235, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.760648
Epoch 160
Validation binary_cross_entropy = 0.759490
Epoch 161
Validation binary_cross_entropy = 0.760736
Epoch 162
Validation binary_cross_entropy = 0.790645
Epoch 163
Validation binary_cross_entropy = 0.818161
Epoch 164
Loss = 6.3569e-02, PNorm = 60.7604, GNorm = 1.5614, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.819275
Epoch 165
Validation binary_cross_entropy = 0.807047
Epoch 166
Validation binary_cross_entropy = 0.831642
Epoch 167
Validation binary_cross_entropy = 0.831541
Epoch 168
Validation binary_cross_entropy = 0.851934
Epoch 169
Loss = 5.3492e-02, PNorm = 60.8673, GNorm = 2.3530, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.897662
Epoch 170
Validation binary_cross_entropy = 0.880408
Epoch 171
Validation binary_cross_entropy = 0.850934
Epoch 172
Validation binary_cross_entropy = 0.966869
Epoch 173
Validation binary_cross_entropy = 0.979031
Epoch 174
Loss = 6.5829e-02, PNorm = 60.9800, GNorm = 4.4684, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.952529
Epoch 175
Validation binary_cross_entropy = 0.960327
Epoch 176
Validation binary_cross_entropy = 0.937551
Epoch 177
Validation binary_cross_entropy = 0.940871
Epoch 178
Validation binary_cross_entropy = 0.926050
Epoch 179
Loss = 5.1239e-02, PNorm = 61.1024, GNorm = 8.6470, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.890232
Epoch 180
Validation binary_cross_entropy = 0.952790
Epoch 181
Validation binary_cross_entropy = 0.973532
Epoch 182
Validation binary_cross_entropy = 0.927603
Epoch 183
Validation binary_cross_entropy = 0.860151
Epoch 184
Loss = 2.6636e-02, PNorm = 61.2523, GNorm = 3.2576, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.874676
Epoch 185
Validation binary_cross_entropy = 0.907389
Epoch 186
Validation binary_cross_entropy = 0.965048
Epoch 187
Validation binary_cross_entropy = 1.023277
Epoch 188
Validation binary_cross_entropy = 0.984003
Epoch 189
Loss = 4.6168e-02, PNorm = 61.3751, GNorm = 2.3585, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.019897
Epoch 190
Validation binary_cross_entropy = 0.968690
Epoch 191
Validation binary_cross_entropy = 1.039170
Epoch 192
Validation binary_cross_entropy = 1.045502
Epoch 193
Validation binary_cross_entropy = 1.056714
Epoch 194
Loss = 1.3376e-01, PNorm = 61.4873, GNorm = 12.0395, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.105466
Epoch 195
Validation binary_cross_entropy = 1.063422
Epoch 196
Validation binary_cross_entropy = 1.096673
Epoch 197
Validation binary_cross_entropy = 1.016189
Epoch 198
Validation binary_cross_entropy = 1.139050
Epoch 199
Loss = 2.7487e-01, PNorm = 61.6373, GNorm = 18.3414, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.876042
Epoch 200
Validation binary_cross_entropy = 0.814236
Epoch 201
Validation binary_cross_entropy = 0.787047
Epoch 202
Validation binary_cross_entropy = 0.758143
Epoch 203
Validation binary_cross_entropy = 0.846880
Epoch 204
Loss = 1.1964e-01, PNorm = 61.8607, GNorm = 21.0341, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.868940
Epoch 205
Validation binary_cross_entropy = 0.948104
Epoch 206
Validation binary_cross_entropy = 1.056560
Epoch 207
Validation binary_cross_entropy = 0.947689
Epoch 208
Validation binary_cross_entropy = 0.867575
Epoch 209
Loss = 4.3014e-02, PNorm = 62.0415, GNorm = 1.0794, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.876155
Epoch 210
Validation binary_cross_entropy = 0.887644
Epoch 211
Validation binary_cross_entropy = 0.807959
Epoch 212
Validation binary_cross_entropy = 0.832828
Epoch 213
Validation binary_cross_entropy = 0.897725
Epoch 214
Loss = 7.9439e-02, PNorm = 62.1781, GNorm = 2.2906, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.871760
Epoch 215
Validation binary_cross_entropy = 0.845906
Epoch 216
Validation binary_cross_entropy = 0.853262
Epoch 217
Validation binary_cross_entropy = 0.893852
Epoch 218
Validation binary_cross_entropy = 0.898477
Epoch 219
Loss = 3.2155e-02, PNorm = 62.2929, GNorm = 1.6749, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.931047
Epoch 220
Validation binary_cross_entropy = 0.967428
Epoch 221
Validation binary_cross_entropy = 0.997032
Epoch 222
Validation binary_cross_entropy = 1.040506
Epoch 223
Validation binary_cross_entropy = 1.017923
Epoch 224
Loss = 6.1482e-02, PNorm = 62.3904, GNorm = 4.3736, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.028722
Epoch 225
Validation binary_cross_entropy = 1.057813
Epoch 226
Validation binary_cross_entropy = 1.072191
Epoch 227
Validation binary_cross_entropy = 1.101149
Epoch 228
Validation binary_cross_entropy = 1.171086
Epoch 229
Loss = 4.7072e-02, PNorm = 62.4830, GNorm = 2.0771, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.165905
Epoch 230
Validation binary_cross_entropy = 1.113348
Epoch 231
Validation binary_cross_entropy = 1.070794
Epoch 232
Validation binary_cross_entropy = 1.052058
Epoch 233
Validation binary_cross_entropy = 1.051174
Epoch 234
Loss = 1.7772e-02, PNorm = 62.5429, GNorm = 0.5085, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.060664
Epoch 235
Validation binary_cross_entropy = 1.078707
Epoch 236
Validation binary_cross_entropy = 1.119078
Epoch 237
Validation binary_cross_entropy = 1.142713
Epoch 238
Validation binary_cross_entropy = 1.118587
Epoch 239
Loss = 4.2050e-02, PNorm = 62.5988, GNorm = 0.7268, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.105328
Epoch 240
Validation binary_cross_entropy = 1.117887
Epoch 241
Validation binary_cross_entropy = 1.138891
Epoch 242
Validation binary_cross_entropy = 1.191851
Epoch 243
Validation binary_cross_entropy = 1.267931
Epoch 244
Loss = 1.4980e-02, PNorm = 62.6623, GNorm = 1.9406, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.286999
Epoch 245
Validation binary_cross_entropy = 1.228125
Epoch 246
Validation binary_cross_entropy = 1.172197
Epoch 247
Validation binary_cross_entropy = 1.159339
Epoch 248
Validation binary_cross_entropy = 1.178245
Epoch 249
Loss = 9.9893e-03, PNorm = 62.7020, GNorm = 1.9249, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.207711
Epoch 250
Validation binary_cross_entropy = 1.214306
Epoch 251
Validation binary_cross_entropy = 1.194705
Epoch 252
Validation binary_cross_entropy = 1.310926
Epoch 253
Validation binary_cross_entropy = 1.222962
Epoch 254
Loss = 1.4167e-02, PNorm = 62.7642, GNorm = 1.1329, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.145740
Epoch 255
Validation binary_cross_entropy = 1.157067
Epoch 256
Validation binary_cross_entropy = 1.143429
Epoch 257
Validation binary_cross_entropy = 1.122619
Epoch 258
Validation binary_cross_entropy = 1.271258
Epoch 259
Loss = 1.0325e-01, PNorm = 62.8718, GNorm = 10.2712, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.328386
Epoch 260
Validation binary_cross_entropy = 1.250024
Epoch 261
Validation binary_cross_entropy = 1.225390
Epoch 262
Validation binary_cross_entropy = 1.244156
Epoch 263
Validation binary_cross_entropy = 1.233842
Epoch 264
Loss = 1.0621e-01, PNorm = 62.9785, GNorm = 7.4187, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.254028
Epoch 265
Validation binary_cross_entropy = 1.340819
Epoch 266
Validation binary_cross_entropy = 1.363417
Epoch 267
Validation binary_cross_entropy = 1.300335
Epoch 268
Validation binary_cross_entropy = 1.248094
Epoch 269
Loss = 1.6401e-02, PNorm = 63.0772, GNorm = 0.9370, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.236042
Epoch 270
Validation binary_cross_entropy = 1.256476
Epoch 271
Validation binary_cross_entropy = 1.267478
Epoch 272
Validation binary_cross_entropy = 1.246610
Epoch 273
Validation binary_cross_entropy = 1.283309
Epoch 274
Loss = 1.4923e-02, PNorm = 63.1559, GNorm = 1.7398, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.325067
Epoch 275
Validation binary_cross_entropy = 1.288208
Epoch 276
Validation binary_cross_entropy = 1.237591
Epoch 277
Validation binary_cross_entropy = 1.204987
Epoch 278
Validation binary_cross_entropy = 1.203411
Epoch 279
Loss = 5.6829e-02, PNorm = 63.2184, GNorm = 7.0589, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.215741
Epoch 280
Validation binary_cross_entropy = 1.242224
Epoch 281
Validation binary_cross_entropy = 1.283419
Epoch 282
Validation binary_cross_entropy = 1.314106
Epoch 283
Validation binary_cross_entropy = 1.329099
Epoch 284
Loss = 5.7909e-03, PNorm = 63.2573, GNorm = 1.6545, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.329641
Epoch 285
Validation binary_cross_entropy = 1.340222
Epoch 286
Validation binary_cross_entropy = 1.353940
Epoch 287
Validation binary_cross_entropy = 1.371928
Epoch 288
Validation binary_cross_entropy = 1.376559
Epoch 289
Loss = 2.9228e-03, PNorm = 63.2853, GNorm = 0.2710, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.378989
Epoch 290
Validation binary_cross_entropy = 1.378569
Epoch 291
Validation binary_cross_entropy = 1.373865
Epoch 292
Validation binary_cross_entropy = 1.372465
Epoch 293
Validation binary_cross_entropy = 1.375696
Epoch 294
Loss = 1.5035e-02, PNorm = 63.3094, GNorm = 0.5022, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.395666
Epoch 295
Validation binary_cross_entropy = 1.419538
Epoch 296
Validation binary_cross_entropy = 1.433878
Epoch 297
Validation binary_cross_entropy = 1.434245
Epoch 298
Validation binary_cross_entropy = 1.414022
Epoch 299
Loss = 5.1732e-03, PNorm = 63.3346, GNorm = 0.3802, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.396830
Model 0 best validation binary_cross_entropy = 0.339055 on epoch 54
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.209651
Ensemble test binary_cross_entropy = 0.209651
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.510514
Epoch 1
Validation binary_cross_entropy = 0.434923
Epoch 2
Validation binary_cross_entropy = 0.605138
Epoch 3
Validation binary_cross_entropy = 0.923352
Epoch 4
Loss = 1.4753e+00, PNorm = 54.1634, GNorm = 28.3003, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.786355
Epoch 5
Validation binary_cross_entropy = 0.517496
Epoch 6
Validation binary_cross_entropy = 0.453328
Epoch 7
Validation binary_cross_entropy = 0.494763
Epoch 8
Validation binary_cross_entropy = 0.674238
Epoch 9
Loss = 1.3263e+00, PNorm = 54.3605, GNorm = 16.5570, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.856285
Epoch 10
Validation binary_cross_entropy = 0.706226
Epoch 11
Validation binary_cross_entropy = 0.460177
Epoch 12
Validation binary_cross_entropy = 0.431394
Epoch 13
Validation binary_cross_entropy = 0.491842
Epoch 14
Loss = 1.1735e+00, PNorm = 54.5407, GNorm = 9.9333, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.654396
Epoch 15
Validation binary_cross_entropy = 0.783365
Epoch 16
Validation binary_cross_entropy = 0.680224
Epoch 17
Validation binary_cross_entropy = 0.494978
Epoch 18
Validation binary_cross_entropy = 0.389165
Epoch 19
Loss = 1.3434e+00, PNorm = 54.6877, GNorm = 20.5699, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.387924
Epoch 20
Validation binary_cross_entropy = 0.447803
Epoch 21
Validation binary_cross_entropy = 0.489454
Epoch 22
Validation binary_cross_entropy = 0.404125
Epoch 23
Validation binary_cross_entropy = 0.371384
Epoch 24
Loss = 1.0042e+00, PNorm = 54.8163, GNorm = 21.3898, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.398328
Epoch 25
Validation binary_cross_entropy = 0.549831
Epoch 26
Validation binary_cross_entropy = 0.682307
Epoch 27
Validation binary_cross_entropy = 0.394875
Epoch 28
Validation binary_cross_entropy = 0.352551
Epoch 29
Loss = 1.4429e+00, PNorm = 54.9382, GNorm = 38.7055, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.342907
Epoch 30
Validation binary_cross_entropy = 0.564867
Epoch 31
Validation binary_cross_entropy = 0.715892
Epoch 32
Validation binary_cross_entropy = 0.547683
Epoch 33
Validation binary_cross_entropy = 0.410794
Epoch 34
Loss = 1.0852e+00, PNorm = 55.0762, GNorm = 12.3888, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.388255
Epoch 35
Validation binary_cross_entropy = 0.391833
Epoch 36
Validation binary_cross_entropy = 0.500054
Epoch 37
Validation binary_cross_entropy = 0.536761
Epoch 38
Validation binary_cross_entropy = 0.385043
Epoch 39
Loss = 1.0033e+00, PNorm = 55.2128, GNorm = 39.1445, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.372797
Epoch 40
Validation binary_cross_entropy = 0.396617
Epoch 41
Validation binary_cross_entropy = 0.529999
Epoch 42
Validation binary_cross_entropy = 0.563411
Epoch 43
Validation binary_cross_entropy = 0.497541
Epoch 44
Loss = 6.8527e-01, PNorm = 55.3884, GNorm = 13.3581, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.363709
Epoch 45
Validation binary_cross_entropy = 0.348735
Epoch 46
Validation binary_cross_entropy = 0.391457
Epoch 47
Validation binary_cross_entropy = 0.487903
Epoch 48
Validation binary_cross_entropy = 0.352430
Epoch 49
Loss = 7.0640e-01, PNorm = 55.5740, GNorm = 13.7785, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.352123
Epoch 50
Validation binary_cross_entropy = 0.404611
Epoch 51
Validation binary_cross_entropy = 0.473927
Epoch 52
Validation binary_cross_entropy = 0.430699
Epoch 53
Validation binary_cross_entropy = 0.401140
Epoch 54
Loss = 6.6064e-01, PNorm = 55.7772, GNorm = 21.5027, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.394423
Epoch 55
Validation binary_cross_entropy = 0.364172
Epoch 56
Validation binary_cross_entropy = 0.359029
Epoch 57
Validation binary_cross_entropy = 0.367651
Epoch 58
Validation binary_cross_entropy = 0.417039
Epoch 59
Loss = 8.3438e-01, PNorm = 55.9563, GNorm = 14.7749, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.467787
Epoch 60
Validation binary_cross_entropy = 0.385824
Epoch 61
Validation binary_cross_entropy = 0.366519
Epoch 62
Validation binary_cross_entropy = 0.434925
Epoch 63
Validation binary_cross_entropy = 0.673454
Epoch 64
Loss = 5.6228e-01, PNorm = 56.1719, GNorm = 14.6898, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.425049
Epoch 65
Validation binary_cross_entropy = 0.366554
Epoch 66
Validation binary_cross_entropy = 0.393513
Epoch 67
Validation binary_cross_entropy = 0.579991
Epoch 68
Validation binary_cross_entropy = 0.482508
Epoch 69
Loss = 5.3538e-01, PNorm = 56.4137, GNorm = 7.4515, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.395525
Epoch 70
Validation binary_cross_entropy = 0.414751
Epoch 71
Validation binary_cross_entropy = 0.398410
Epoch 72
Validation binary_cross_entropy = 0.429782
Epoch 73
Validation binary_cross_entropy = 0.584650
Epoch 74
Loss = 3.8915e-01, PNorm = 56.6781, GNorm = 12.9246, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.481160
Epoch 75
Validation binary_cross_entropy = 0.610028
Epoch 76
Validation binary_cross_entropy = 0.473737
Epoch 77
Validation binary_cross_entropy = 0.410265
Epoch 78
Validation binary_cross_entropy = 0.497067
Epoch 79
Loss = 5.9702e-01, PNorm = 56.9548, GNorm = 30.0736, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.529607
Epoch 80
Validation binary_cross_entropy = 0.424441
Epoch 81
Validation binary_cross_entropy = 0.465970
Epoch 82
Validation binary_cross_entropy = 0.548049
Epoch 83
Validation binary_cross_entropy = 0.455903
Epoch 84
Loss = 6.3221e-01, PNorm = 57.2923, GNorm = 10.4058, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.419478
Epoch 85
Validation binary_cross_entropy = 0.406853
Epoch 86
Validation binary_cross_entropy = 0.431121
Epoch 87
Validation binary_cross_entropy = 0.411118
Epoch 88
Validation binary_cross_entropy = 0.437345
Epoch 89
Loss = 5.1224e-01, PNorm = 57.6181, GNorm = 7.3660, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.434927
Epoch 90
Validation binary_cross_entropy = 0.498327
Epoch 91
Validation binary_cross_entropy = 0.594097
Epoch 92
Validation binary_cross_entropy = 0.471612
Epoch 93
Validation binary_cross_entropy = 0.422697
Epoch 94
Loss = 3.8634e-01, PNorm = 57.9323, GNorm = 5.4697, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.498896
Epoch 95
Validation binary_cross_entropy = 0.642768
Epoch 96
Validation binary_cross_entropy = 0.480671
Epoch 97
Validation binary_cross_entropy = 0.419729
Epoch 98
Validation binary_cross_entropy = 0.473239
Epoch 99
Loss = 4.1826e-01, PNorm = 58.2511, GNorm = 5.3443, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.446340
Epoch 100
Validation binary_cross_entropy = 0.484614
Epoch 101
Validation binary_cross_entropy = 0.563070
Epoch 102
Validation binary_cross_entropy = 0.493007
Epoch 103
Validation binary_cross_entropy = 0.566563
Epoch 104
Loss = 3.4084e-01, PNorm = 58.5475, GNorm = 13.9595, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.545024
Epoch 105
Validation binary_cross_entropy = 0.549744
Epoch 106
Validation binary_cross_entropy = 0.499797
Epoch 107
Validation binary_cross_entropy = 0.496877
Epoch 108
Validation binary_cross_entropy = 0.509542
Epoch 109
Loss = 2.1521e-01, PNorm = 58.8501, GNorm = 12.4686, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.544378
Epoch 110
Validation binary_cross_entropy = 0.682023
Epoch 111
Validation binary_cross_entropy = 0.582970
Epoch 112
Validation binary_cross_entropy = 0.547005
Epoch 113
Validation binary_cross_entropy = 0.574376
Epoch 114
Loss = 1.6575e-01, PNorm = 59.1449, GNorm = 9.0053, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.593104
Epoch 115
Validation binary_cross_entropy = 0.565510
Epoch 116
Validation binary_cross_entropy = 0.550186
Epoch 117
Validation binary_cross_entropy = 0.492389
Epoch 118
Validation binary_cross_entropy = 0.497937
Epoch 119
Loss = 1.8150e-01, PNorm = 59.4311, GNorm = 8.7088, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.505806
Epoch 120
Validation binary_cross_entropy = 0.540731
Epoch 121
Validation binary_cross_entropy = 0.539901
Epoch 122
Validation binary_cross_entropy = 0.553145
Epoch 123
Validation binary_cross_entropy = 0.571886
Epoch 124
Loss = 1.4206e-01, PNorm = 59.7067, GNorm = 6.5790, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.610144
Epoch 125
Validation binary_cross_entropy = 0.664304
Epoch 126
Validation binary_cross_entropy = 0.624900
Epoch 127
Validation binary_cross_entropy = 0.645623
Epoch 128
Validation binary_cross_entropy = 0.609608
Epoch 129
Loss = 9.3245e-02, PNorm = 59.9536, GNorm = 11.3320, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.644229
Epoch 130
Validation binary_cross_entropy = 0.632522
Epoch 131
Validation binary_cross_entropy = 0.678879
Epoch 132
Validation binary_cross_entropy = 0.690379
Epoch 133
Validation binary_cross_entropy = 0.665488
Epoch 134
Loss = 9.0080e-02, PNorm = 60.1752, GNorm = 2.9027, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.692363
Epoch 135
Validation binary_cross_entropy = 0.688740
Epoch 136
Validation binary_cross_entropy = 0.624841
Epoch 137
Validation binary_cross_entropy = 0.655946
Epoch 138
Validation binary_cross_entropy = 0.653944
Epoch 139
Loss = 1.4014e-01, PNorm = 60.3906, GNorm = 13.7102, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.577340
Epoch 140
Validation binary_cross_entropy = 0.576675
Epoch 141
Validation binary_cross_entropy = 0.625617
Epoch 142
Validation binary_cross_entropy = 0.710833
Epoch 143
Validation binary_cross_entropy = 0.758528
Epoch 144
Loss = 2.1604e-01, PNorm = 60.5970, GNorm = 17.8572, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.760530
Epoch 145
Validation binary_cross_entropy = 0.817070
Epoch 146
Validation binary_cross_entropy = 0.756941
Epoch 147
Validation binary_cross_entropy = 0.731935
Epoch 148
Validation binary_cross_entropy = 0.712456
Epoch 149
Loss = 1.6752e-01, PNorm = 60.7993, GNorm = 6.1221, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.652473
Epoch 150
Validation binary_cross_entropy = 0.670257
Epoch 151
Validation binary_cross_entropy = 0.684223
Epoch 152
Validation binary_cross_entropy = 0.687662
Epoch 153
Validation binary_cross_entropy = 0.688773
Epoch 154
Loss = 1.1230e-01, PNorm = 61.0194, GNorm = 3.5160, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.699323
Epoch 155
Validation binary_cross_entropy = 0.724473
Epoch 156
Validation binary_cross_entropy = 0.770210
Epoch 157
Validation binary_cross_entropy = 0.788106
Epoch 158
Validation binary_cross_entropy = 0.796945
Epoch 159
Loss = 9.7379e-02, PNorm = 61.1795, GNorm = 4.2921, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.825371
Epoch 160
Validation binary_cross_entropy = 0.874291
Epoch 161
Validation binary_cross_entropy = 0.873599
Epoch 162
Validation binary_cross_entropy = 0.942850
Epoch 163
Validation binary_cross_entropy = 0.818446
Epoch 164
Loss = 5.0147e-02, PNorm = 61.3545, GNorm = 1.8511, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.779096
Epoch 165
Validation binary_cross_entropy = 0.735763
Epoch 166
Validation binary_cross_entropy = 0.750147
Epoch 167
Validation binary_cross_entropy = 0.743545
Epoch 168
Validation binary_cross_entropy = 0.759539
Epoch 169
Loss = 1.0923e-01, PNorm = 61.5287, GNorm = 3.5387, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.786591
Epoch 170
Validation binary_cross_entropy = 0.821301
Epoch 171
Validation binary_cross_entropy = 0.867729
Epoch 172
Validation binary_cross_entropy = 0.944571
Epoch 173
Validation binary_cross_entropy = 0.988774
Epoch 174
Loss = 3.8119e-02, PNorm = 61.6726, GNorm = 4.3283, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.935809
Epoch 175
Validation binary_cross_entropy = 0.908468
Epoch 176
Validation binary_cross_entropy = 0.914527
Epoch 177
Validation binary_cross_entropy = 0.963621
Epoch 178
Validation binary_cross_entropy = 1.008058
Epoch 179
Loss = 4.1399e-02, PNorm = 61.7916, GNorm = 1.7447, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.942771
Epoch 180
Validation binary_cross_entropy = 0.923550
Epoch 181
Validation binary_cross_entropy = 0.928066
Epoch 182
Validation binary_cross_entropy = 0.975320
Epoch 183
Validation binary_cross_entropy = 1.079615
Epoch 184
Loss = 8.0949e-02, PNorm = 61.8903, GNorm = 10.8798, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.061527
Epoch 185
Validation binary_cross_entropy = 0.995605
Epoch 186
Validation binary_cross_entropy = 0.995889
Epoch 187
Validation binary_cross_entropy = 0.986112
Epoch 188
Validation binary_cross_entropy = 0.948348
Epoch 189
Loss = 1.0432e-01, PNorm = 62.0082, GNorm = 7.0560, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.942900
Epoch 190
Validation binary_cross_entropy = 0.966915
Epoch 191
Validation binary_cross_entropy = 0.971932
Epoch 192
Validation binary_cross_entropy = 0.932096
Epoch 193
Validation binary_cross_entropy = 0.923821
Epoch 194
Loss = 6.4680e-02, PNorm = 62.1211, GNorm = 3.6805, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.945672
Epoch 195
Validation binary_cross_entropy = 1.002147
Epoch 196
Validation binary_cross_entropy = 1.038131
Epoch 197
Validation binary_cross_entropy = 1.059077
Epoch 198
Validation binary_cross_entropy = 1.306214
Epoch 199
Loss = 2.8948e-02, PNorm = 62.2292, GNorm = 0.4647, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.675665
Epoch 200
Validation binary_cross_entropy = 1.227896
Epoch 201
Validation binary_cross_entropy = 1.050587
Epoch 202
Validation binary_cross_entropy = 1.019698
Epoch 203
Validation binary_cross_entropy = 0.891502
Epoch 204
Loss = 1.9284e-02, PNorm = 62.3406, GNorm = 0.9456, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.864285
Epoch 205
Validation binary_cross_entropy = 0.859972
Epoch 206
Validation binary_cross_entropy = 0.962172
Epoch 207
Validation binary_cross_entropy = 1.036130
Epoch 208
Validation binary_cross_entropy = 1.036917
Epoch 209
Loss = 4.0805e-02, PNorm = 62.4794, GNorm = 4.0200, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.064208
Epoch 210
Validation binary_cross_entropy = 1.098380
Epoch 211
Validation binary_cross_entropy = 1.104009
Epoch 212
Validation binary_cross_entropy = 1.121117
Epoch 213
Validation binary_cross_entropy = 1.098406
Epoch 214
Loss = 3.9750e-02, PNorm = 62.5920, GNorm = 2.3050, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.060007
Epoch 215
Validation binary_cross_entropy = 1.043748
Epoch 216
Validation binary_cross_entropy = 1.029064
Epoch 217
Validation binary_cross_entropy = 1.044798
Epoch 218
Validation binary_cross_entropy = 1.046052
Epoch 219
Loss = 3.9016e-02, PNorm = 62.7089, GNorm = 0.8400, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.057400
Epoch 220
Validation binary_cross_entropy = 1.084009
Epoch 221
Validation binary_cross_entropy = 1.142159
Epoch 222
Validation binary_cross_entropy = 1.232313
Epoch 223
Validation binary_cross_entropy = 1.138609
Epoch 224
Loss = 2.7788e-02, PNorm = 62.8118, GNorm = 4.9643, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.108490
Epoch 225
Validation binary_cross_entropy = 1.119047
Epoch 226
Validation binary_cross_entropy = 1.118915
Epoch 227
Validation binary_cross_entropy = 1.124477
Epoch 228
Validation binary_cross_entropy = 1.116231
Epoch 229
Loss = 2.1791e-02, PNorm = 62.8992, GNorm = 1.8334, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.117394
Epoch 230
Validation binary_cross_entropy = 1.125752
Epoch 231
Validation binary_cross_entropy = 1.149601
Epoch 232
Validation binary_cross_entropy = 1.139420
Epoch 233
Validation binary_cross_entropy = 1.160915
Epoch 234
Loss = 1.3192e-02, PNorm = 62.9551, GNorm = 2.0868, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.174414
Epoch 235
Validation binary_cross_entropy = 1.183914
Epoch 236
Validation binary_cross_entropy = 1.262505
Epoch 237
Validation binary_cross_entropy = 1.269457
Epoch 238
Validation binary_cross_entropy = 1.210492
Epoch 239
Loss = 5.2665e-02, PNorm = 63.0283, GNorm = 11.2539, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.159928
Epoch 240
Validation binary_cross_entropy = 1.185744
Epoch 241
Validation binary_cross_entropy = 1.304570
Epoch 242
Validation binary_cross_entropy = 1.295997
Epoch 243
Validation binary_cross_entropy = 1.216287
Epoch 244
Loss = 2.0715e-02, PNorm = 63.1309, GNorm = 3.1345, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.171974
Epoch 245
Validation binary_cross_entropy = 1.146521
Epoch 246
Validation binary_cross_entropy = 1.136160
Epoch 247
Validation binary_cross_entropy = 1.129180
Epoch 248
Validation binary_cross_entropy = 1.133358
Epoch 249
Loss = 1.6973e-02, PNorm = 63.2237, GNorm = 0.1539, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.154637
Epoch 250
Validation binary_cross_entropy = 1.164883
Epoch 251
Validation binary_cross_entropy = 1.180961
Epoch 252
Validation binary_cross_entropy = 1.177547
Epoch 253
Validation binary_cross_entropy = 1.174638
Epoch 254
Loss = 1.1329e-02, PNorm = 63.3124, GNorm = 0.5174, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.174313
Epoch 255
Validation binary_cross_entropy = 1.180094
Epoch 256
Validation binary_cross_entropy = 1.189988
Epoch 257
Validation binary_cross_entropy = 1.198941
Epoch 258
Validation binary_cross_entropy = 1.207827
Epoch 259
Loss = 3.3125e-02, PNorm = 63.3754, GNorm = 3.7229, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.218751
Epoch 260
Validation binary_cross_entropy = 1.287600
Epoch 261
Validation binary_cross_entropy = 1.376320
Epoch 262
Validation binary_cross_entropy = 1.312575
Epoch 263
Validation binary_cross_entropy = 1.188980
Epoch 264
Loss = 9.6449e-03, PNorm = 63.4336, GNorm = 1.7809, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.207188
Epoch 265
Validation binary_cross_entropy = 1.131652
Epoch 266
Validation binary_cross_entropy = 1.119880
Epoch 267
Validation binary_cross_entropy = 1.178469
Epoch 268
Validation binary_cross_entropy = 1.246453
Epoch 269
Loss = 5.6806e-03, PNorm = 63.5100, GNorm = 0.8842, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.295962
Epoch 270
Validation binary_cross_entropy = 1.244590
Epoch 271
Validation binary_cross_entropy = 1.204678
Epoch 272
Validation binary_cross_entropy = 1.264260
Epoch 273
Validation binary_cross_entropy = 1.245281
Epoch 274
Loss = 3.7522e-02, PNorm = 63.5786, GNorm = 0.2794, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.285914
Epoch 275
Validation binary_cross_entropy = 1.296274
Epoch 276
Validation binary_cross_entropy = 1.269083
Epoch 277
Validation binary_cross_entropy = 1.259299
Epoch 278
Validation binary_cross_entropy = 1.227789
Epoch 279
Loss = 4.0795e-03, PNorm = 63.6715, GNorm = 0.3326, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.216662
Epoch 280
Validation binary_cross_entropy = 1.210851
Epoch 281
Validation binary_cross_entropy = 1.182943
Epoch 282
Validation binary_cross_entropy = 1.197851
Epoch 283
Validation binary_cross_entropy = 1.211120
Epoch 284
Loss = 5.7301e-03, PNorm = 63.7302, GNorm = 1.1160, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.226433
Epoch 285
Validation binary_cross_entropy = 1.255462
Epoch 286
Validation binary_cross_entropy = 1.318860
Epoch 287
Validation binary_cross_entropy = 1.397823
Epoch 288
Validation binary_cross_entropy = 1.417058
Epoch 289
Loss = 7.0763e-02, PNorm = 63.7928, GNorm = 0.6851, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.371720
Epoch 290
Validation binary_cross_entropy = 1.351532
Epoch 291
Validation binary_cross_entropy = 1.306659
Epoch 292
Validation binary_cross_entropy = 1.254952
Epoch 293
Validation binary_cross_entropy = 1.367348
Epoch 294
Loss = 7.5907e-02, PNorm = 63.8572, GNorm = 9.0316, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.465399
Epoch 295
Validation binary_cross_entropy = 1.374816
Epoch 296
Validation binary_cross_entropy = 1.335331
Epoch 297
Validation binary_cross_entropy = 1.361365
Epoch 298
Validation binary_cross_entropy = 1.357646
Epoch 299
Loss = 5.1523e-02, PNorm = 63.9678, GNorm = 7.1095, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.345117
Model 0 best validation binary_cross_entropy = 0.342907 on epoch 29
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.231419
Ensemble test binary_cross_entropy = 0.231419
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.501251
Epoch 1
Validation binary_cross_entropy = 0.430462
Epoch 2
Validation binary_cross_entropy = 0.575533
Epoch 3
Validation binary_cross_entropy = 0.815033
Epoch 4
Loss = 1.4338e+00, PNorm = 54.1623, GNorm = 24.5260, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.714511
Epoch 5
Validation binary_cross_entropy = 0.484576
Epoch 6
Validation binary_cross_entropy = 0.421411
Epoch 7
Validation binary_cross_entropy = 0.457900
Epoch 8
Validation binary_cross_entropy = 0.555449
Epoch 9
Loss = 1.7131e+00, PNorm = 54.3544, GNorm = 18.9963, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.864399
Epoch 10
Validation binary_cross_entropy = 0.718619
Epoch 11
Validation binary_cross_entropy = 0.537177
Epoch 12
Validation binary_cross_entropy = 0.450682
Epoch 13
Validation binary_cross_entropy = 0.459448
Epoch 14
Loss = 1.6640e+00, PNorm = 54.5294, GNorm = 18.8201, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.517334
Epoch 15
Validation binary_cross_entropy = 0.761388
Epoch 16
Validation binary_cross_entropy = 0.912406
Epoch 17
Validation binary_cross_entropy = 0.695941
Epoch 18
Validation binary_cross_entropy = 0.440827
Epoch 19
Loss = 8.9702e-01, PNorm = 54.6687, GNorm = 8.3975, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.378406
Epoch 20
Validation binary_cross_entropy = 0.430207
Epoch 21
Validation binary_cross_entropy = 0.524032
Epoch 22
Validation binary_cross_entropy = 0.638753
Epoch 23
Validation binary_cross_entropy = 0.490564
Epoch 24
Loss = 1.1358e+00, PNorm = 54.7942, GNorm = 11.0696, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.434231
Epoch 25
Validation binary_cross_entropy = 0.381456
Epoch 26
Validation binary_cross_entropy = 0.388471
Epoch 27
Validation binary_cross_entropy = 0.511986
Epoch 28
Validation binary_cross_entropy = 0.626537
Epoch 29
Loss = 1.0468e+00, PNorm = 54.9433, GNorm = 13.4807, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.434243
Epoch 30
Validation binary_cross_entropy = 0.370057
Epoch 31
Validation binary_cross_entropy = 0.343504
Epoch 32
Validation binary_cross_entropy = 0.361034
Epoch 33
Validation binary_cross_entropy = 0.399145
Epoch 34
Loss = 9.8426e-01, PNorm = 55.1084, GNorm = 31.1278, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.398926
Epoch 35
Validation binary_cross_entropy = 0.383841
Epoch 36
Validation binary_cross_entropy = 0.416854
Epoch 37
Validation binary_cross_entropy = 0.396273
Epoch 38
Validation binary_cross_entropy = 0.509314
Epoch 39
Loss = 8.1221e-01, PNorm = 55.2970, GNorm = 13.1535, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.378230
Epoch 40
Validation binary_cross_entropy = 0.521844
Epoch 41
Validation binary_cross_entropy = 0.452116
Epoch 42
Validation binary_cross_entropy = 0.423883
Epoch 43
Validation binary_cross_entropy = 0.360677
Epoch 44
Loss = 1.0928e+00, PNorm = 55.4511, GNorm = 10.9998, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.406987
Epoch 45
Validation binary_cross_entropy = 0.439223
Epoch 46
Validation binary_cross_entropy = 0.355596
Epoch 47
Validation binary_cross_entropy = 0.355436
Epoch 48
Validation binary_cross_entropy = 0.471827
Epoch 49
Loss = 9.4335e-01, PNorm = 55.6435, GNorm = 30.7509, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.381198
Epoch 50
Validation binary_cross_entropy = 0.357863
Epoch 51
Validation binary_cross_entropy = 0.374788
Epoch 52
Validation binary_cross_entropy = 0.511746
Epoch 53
Validation binary_cross_entropy = 0.759282
Epoch 54
Loss = 1.0701e+00, PNorm = 55.8336, GNorm = 13.6221, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.563528
Epoch 55
Validation binary_cross_entropy = 0.396001
Epoch 56
Validation binary_cross_entropy = 0.376544
Epoch 57
Validation binary_cross_entropy = 0.396645
Epoch 58
Validation binary_cross_entropy = 0.415017
Epoch 59
Loss = 5.6800e-01, PNorm = 56.0512, GNorm = 15.8878, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.401824
Epoch 60
Validation binary_cross_entropy = 0.389167
Epoch 61
Validation binary_cross_entropy = 0.431464
Epoch 62
Validation binary_cross_entropy = 0.442429
Epoch 63
Validation binary_cross_entropy = 0.394034
Epoch 64
Loss = 5.9624e-01, PNorm = 56.2334, GNorm = 7.2329, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.496889
Epoch 65
Validation binary_cross_entropy = 0.401805
Epoch 66
Validation binary_cross_entropy = 0.425259
Epoch 67
Validation binary_cross_entropy = 0.454489
Epoch 68
Validation binary_cross_entropy = 0.462976
Epoch 69
Loss = 5.0603e-01, PNorm = 56.4455, GNorm = 17.6122, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.547667
Epoch 70
Validation binary_cross_entropy = 0.425718
Epoch 71
Validation binary_cross_entropy = 0.468426
Epoch 72
Validation binary_cross_entropy = 0.665929
Epoch 73
Validation binary_cross_entropy = 0.506748
Epoch 74
Loss = 3.1477e-01, PNorm = 56.6734, GNorm = 7.1544, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.434014
Epoch 75
Validation binary_cross_entropy = 0.434644
Epoch 76
Validation binary_cross_entropy = 0.451274
Epoch 77
Validation binary_cross_entropy = 0.494932
Epoch 78
Validation binary_cross_entropy = 0.454134
Epoch 79
Loss = 4.2924e-01, PNorm = 56.9560, GNorm = 6.0473, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.455055
Epoch 80
Validation binary_cross_entropy = 0.472984
Epoch 81
Validation binary_cross_entropy = 0.482860
Epoch 82
Validation binary_cross_entropy = 0.496059
Epoch 83
Validation binary_cross_entropy = 0.681362
Epoch 84
Loss = 6.9408e-01, PNorm = 57.2581, GNorm = 23.7081, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.452777
Epoch 85
Validation binary_cross_entropy = 0.447267
Epoch 86
Validation binary_cross_entropy = 0.486848
Epoch 87
Validation binary_cross_entropy = 0.445736
Epoch 88
Validation binary_cross_entropy = 0.464547
Epoch 89
Loss = 3.3839e-01, PNorm = 57.5453, GNorm = 23.4921, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.628622
Epoch 90
Validation binary_cross_entropy = 0.497783
Epoch 91
Validation binary_cross_entropy = 0.499487
Epoch 92
Validation binary_cross_entropy = 0.563366
Epoch 93
Validation binary_cross_entropy = 0.642214
Epoch 94
Loss = 2.3592e-01, PNorm = 57.8586, GNorm = 7.8550, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.630195
Epoch 95
Validation binary_cross_entropy = 0.607093
Epoch 96
Validation binary_cross_entropy = 0.602209
Epoch 97
Validation binary_cross_entropy = 0.625217
Epoch 98
Validation binary_cross_entropy = 0.611601
Epoch 99
Loss = 2.7751e-01, PNorm = 58.1328, GNorm = 5.4414, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.636067
Epoch 100
Validation binary_cross_entropy = 0.707409
Epoch 101
Validation binary_cross_entropy = 0.794965
Epoch 102
Validation binary_cross_entropy = 0.787793
Epoch 103
Validation binary_cross_entropy = 0.860865
Epoch 104
Loss = 4.0542e-01, PNorm = 58.4399, GNorm = 19.7431, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.641280
Epoch 105
Validation binary_cross_entropy = 0.639690
Epoch 106
Validation binary_cross_entropy = 0.619832
Epoch 107
Validation binary_cross_entropy = 0.767447
Epoch 108
Validation binary_cross_entropy = 0.558760
Epoch 109
Loss = 2.5260e-01, PNorm = 58.6806, GNorm = 14.2384, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.549062
Epoch 110
Validation binary_cross_entropy = 0.512416
Epoch 111
Validation binary_cross_entropy = 0.568272
Epoch 112
Validation binary_cross_entropy = 0.596504
Epoch 113
Validation binary_cross_entropy = 0.630233
Epoch 114
Loss = 1.1246e-01, PNorm = 58.8997, GNorm = 9.6479, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.696000
Epoch 115
Validation binary_cross_entropy = 0.745827
Epoch 116
Validation binary_cross_entropy = 0.758252
Epoch 117
Validation binary_cross_entropy = 0.740710
Epoch 118
Validation binary_cross_entropy = 0.686427
Epoch 119
Loss = 1.6768e-01, PNorm = 59.0906, GNorm = 4.6751, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.713915
Epoch 120
Validation binary_cross_entropy = 0.656355
Epoch 121
Validation binary_cross_entropy = 0.674588
Epoch 122
Validation binary_cross_entropy = 0.757336
Epoch 123
Validation binary_cross_entropy = 0.758485
Epoch 124
Loss = 2.9210e-01, PNorm = 59.3511, GNorm = 4.5793, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.760137
Epoch 125
Validation binary_cross_entropy = 0.741018
Epoch 126
Validation binary_cross_entropy = 0.737599
Epoch 127
Validation binary_cross_entropy = 0.737690
Epoch 128
Validation binary_cross_entropy = 0.764738
Epoch 129
Loss = 9.8939e-02, PNorm = 59.6011, GNorm = 3.4640, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.819351
Epoch 130
Validation binary_cross_entropy = 0.812021
Epoch 131
Validation binary_cross_entropy = 0.811898
Epoch 132
Validation binary_cross_entropy = 0.781033
Epoch 133
Validation binary_cross_entropy = 0.860297
Epoch 134
Loss = 7.1286e-02, PNorm = 59.7973, GNorm = 14.1116, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.091138
Epoch 135
Validation binary_cross_entropy = 0.731644
Epoch 136
Validation binary_cross_entropy = 0.880246
Epoch 137
Validation binary_cross_entropy = 0.686539
Epoch 138
Validation binary_cross_entropy = 0.745480
Epoch 139
Loss = 3.1188e-01, PNorm = 60.0112, GNorm = 21.8837, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.849394
Epoch 140
Validation binary_cross_entropy = 0.598292
Epoch 141
Validation binary_cross_entropy = 0.608962
Epoch 142
Validation binary_cross_entropy = 0.679371
Epoch 143
Validation binary_cross_entropy = 0.703853
Epoch 144
Loss = 2.4395e-01, PNorm = 60.2711, GNorm = 7.5561, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.823903
Epoch 145
Validation binary_cross_entropy = 1.019853
Epoch 146
Validation binary_cross_entropy = 0.881118
Epoch 147
Validation binary_cross_entropy = 0.921082
Epoch 148
Validation binary_cross_entropy = 0.860647
Epoch 149
Loss = 1.7178e-01, PNorm = 60.4963, GNorm = 8.1683, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.777587
Epoch 150
Validation binary_cross_entropy = 0.735672
Epoch 151
Validation binary_cross_entropy = 0.707561
Epoch 152
Validation binary_cross_entropy = 0.717159
Epoch 153
Validation binary_cross_entropy = 0.811041
Epoch 154
Loss = 9.4657e-02, PNorm = 60.6864, GNorm = 4.9287, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.857905
Epoch 155
Validation binary_cross_entropy = 0.895433
Epoch 156
Validation binary_cross_entropy = 0.961925
Epoch 157
Validation binary_cross_entropy = 0.967632
Epoch 158
Validation binary_cross_entropy = 0.989237
Epoch 159
Loss = 6.5748e-02, PNorm = 60.8455, GNorm = 2.5690, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.034277
Epoch 160
Validation binary_cross_entropy = 1.058490
Epoch 161
Validation binary_cross_entropy = 1.072086
Epoch 162
Validation binary_cross_entropy = 1.144709
Epoch 163
Validation binary_cross_entropy = 1.098004
Epoch 164
Loss = 1.1083e-01, PNorm = 60.9710, GNorm = 2.0495, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.035192
Epoch 165
Validation binary_cross_entropy = 1.085867
Epoch 166
Validation binary_cross_entropy = 1.148719
Epoch 167
Validation binary_cross_entropy = 1.088475
Epoch 168
Validation binary_cross_entropy = 1.116754
Epoch 169
Loss = 1.1837e-01, PNorm = 61.0999, GNorm = 6.9745, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.161340
Epoch 170
Validation binary_cross_entropy = 1.159976
Epoch 171
Validation binary_cross_entropy = 1.195724
Epoch 172
Validation binary_cross_entropy = 1.145907
Epoch 173
Validation binary_cross_entropy = 1.144189
Epoch 174
Loss = 9.4427e-02, PNorm = 61.2374, GNorm = 1.9595, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.084460
Epoch 175
Validation binary_cross_entropy = 1.089718
Epoch 176
Validation binary_cross_entropy = 1.090153
Epoch 177
Validation binary_cross_entropy = 1.083577
Epoch 178
Validation binary_cross_entropy = 1.114991
Epoch 179
Loss = 5.3891e-02, PNorm = 61.3648, GNorm = 0.6840, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.154245
Epoch 180
Validation binary_cross_entropy = 1.197181
Epoch 181
Validation binary_cross_entropy = 1.239076
Epoch 182
Validation binary_cross_entropy = 1.258820
Epoch 183
Validation binary_cross_entropy = 1.283794
Epoch 184
Loss = 7.0906e-02, PNorm = 61.4509, GNorm = 5.7102, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.288741
Epoch 185
Validation binary_cross_entropy = 1.175765
Epoch 186
Validation binary_cross_entropy = 1.227746
Epoch 187
Validation binary_cross_entropy = 1.164234
Epoch 188
Validation binary_cross_entropy = 1.150698
Epoch 189
Loss = 7.6071e-02, PNorm = 61.5567, GNorm = 3.5402, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.235308
Epoch 190
Validation binary_cross_entropy = 1.261948
Epoch 191
Validation binary_cross_entropy = 1.249571
Epoch 192
Validation binary_cross_entropy = 1.250184
Epoch 193
Validation binary_cross_entropy = 1.291130
Epoch 194
Loss = 6.3927e-02, PNorm = 61.6881, GNorm = 3.4350, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.317379
Epoch 195
Validation binary_cross_entropy = 1.340713
Epoch 196
Validation binary_cross_entropy = 1.399475
Epoch 197
Validation binary_cross_entropy = 1.365653
Epoch 198
Validation binary_cross_entropy = 1.372463
Epoch 199
Loss = 5.6856e-02, PNorm = 61.7915, GNorm = 1.0366, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.366007
Epoch 200
Validation binary_cross_entropy = 1.372037
Epoch 201
Validation binary_cross_entropy = 1.358858
Epoch 202
Validation binary_cross_entropy = 1.280015
Epoch 203
Validation binary_cross_entropy = 1.110591
Epoch 204
Loss = 3.1490e-02, PNorm = 61.8984, GNorm = 2.4556, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.062934
Epoch 205
Validation binary_cross_entropy = 1.082521
Epoch 206
Validation binary_cross_entropy = 1.074570
Epoch 207
Validation binary_cross_entropy = 1.190851
Epoch 208
Validation binary_cross_entropy = 1.122419
Epoch 209
Loss = 2.4773e-02, PNorm = 62.0505, GNorm = 0.2472, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.087110
Epoch 210
Validation binary_cross_entropy = 1.296666
Epoch 211
Validation binary_cross_entropy = 1.437431
Epoch 212
Validation binary_cross_entropy = 1.294716
Epoch 213
Validation binary_cross_entropy = 1.111385
Epoch 214
Loss = 7.2716e-02, PNorm = 62.2062, GNorm = 7.5285, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.117747
Epoch 215
Validation binary_cross_entropy = 1.104495
Epoch 216
Validation binary_cross_entropy = 1.104768
Epoch 217
Validation binary_cross_entropy = 1.124277
Epoch 218
Validation binary_cross_entropy = 1.155459
Epoch 219
Loss = 1.0323e-01, PNorm = 62.3327, GNorm = 3.3211, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.193870
Epoch 220
Validation binary_cross_entropy = 1.256999
Epoch 221
Validation binary_cross_entropy = 1.332345
Epoch 222
Validation binary_cross_entropy = 1.339420
Epoch 223
Validation binary_cross_entropy = 1.364968
Epoch 224
Loss = 2.1579e-02, PNorm = 62.4812, GNorm = 1.2992, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.410247
Epoch 225
Validation binary_cross_entropy = 1.442418
Epoch 226
Validation binary_cross_entropy = 1.452933
Epoch 227
Validation binary_cross_entropy = 1.456786
Epoch 228
Validation binary_cross_entropy = 1.452602
Epoch 229
Loss = 1.2059e-02, PNorm = 62.5876, GNorm = 0.9981, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.439729
Epoch 230
Validation binary_cross_entropy = 1.404261
Epoch 231
Validation binary_cross_entropy = 1.346263
Epoch 232
Validation binary_cross_entropy = 1.292151
Epoch 233
Validation binary_cross_entropy = 1.328782
Epoch 234
Loss = 3.0846e-02, PNorm = 62.6623, GNorm = 5.6439, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.456149
Epoch 235
Validation binary_cross_entropy = 1.340290
Epoch 236
Validation binary_cross_entropy = 1.316405
Epoch 237
Validation binary_cross_entropy = 1.351895
Epoch 238
Validation binary_cross_entropy = 1.372864
Epoch 239
Loss = 3.5004e-02, PNorm = 62.7743, GNorm = 5.8736, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.422727
Epoch 240
Validation binary_cross_entropy = 1.467201
Epoch 241
Validation binary_cross_entropy = 1.510062
Epoch 242
Validation binary_cross_entropy = 1.591119
Epoch 243
Validation binary_cross_entropy = 1.617603
Epoch 244
Loss = 7.1020e-02, PNorm = 62.8900, GNorm = 3.7600, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.548386
Epoch 245
Validation binary_cross_entropy = 1.464074
Epoch 246
Validation binary_cross_entropy = 1.397729
Epoch 247
Validation binary_cross_entropy = 1.348724
Epoch 248
Validation binary_cross_entropy = 1.342916
Epoch 249
Loss = 4.9293e-02, PNorm = 62.9721, GNorm = 0.4528, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.326015
Epoch 250
Validation binary_cross_entropy = 1.361878
Epoch 251
Validation binary_cross_entropy = 1.412843
Epoch 252
Validation binary_cross_entropy = 1.422328
Epoch 253
Validation binary_cross_entropy = 1.416931
Epoch 254
Loss = 2.6521e-02, PNorm = 63.0208, GNorm = 3.3214, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.432275
Epoch 255
Validation binary_cross_entropy = 1.420337
Epoch 256
Validation binary_cross_entropy = 1.438388
Epoch 257
Validation binary_cross_entropy = 1.467898
Epoch 258
Validation binary_cross_entropy = 1.494133
Epoch 259
Loss = 3.0785e-03, PNorm = 63.0746, GNorm = 0.2230, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.519808
Epoch 260
Validation binary_cross_entropy = 1.558051
Epoch 261
Validation binary_cross_entropy = 1.610450
Epoch 262
Validation binary_cross_entropy = 1.652006
Epoch 263
Validation binary_cross_entropy = 1.675563
Epoch 264
Loss = 1.1585e-02, PNorm = 63.1289, GNorm = 2.0492, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.655619
Epoch 265
Validation binary_cross_entropy = 1.618566
Epoch 266
Validation binary_cross_entropy = 1.607604
Epoch 267
Validation binary_cross_entropy = 1.614994
Epoch 268
Validation binary_cross_entropy = 1.651395
Epoch 269
Loss = 7.4148e-03, PNorm = 63.1772, GNorm = 0.7088, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.685003
Epoch 270
Validation binary_cross_entropy = 1.712834
Epoch 271
Validation binary_cross_entropy = 1.739991
Epoch 272
Validation binary_cross_entropy = 1.742771
Epoch 273
Validation binary_cross_entropy = 1.719804
Epoch 274
Loss = 5.7882e-03, PNorm = 63.2159, GNorm = 0.1352, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.702018
Epoch 275
Validation binary_cross_entropy = 1.693120
Epoch 276
Validation binary_cross_entropy = 1.692164
Epoch 277
Validation binary_cross_entropy = 1.727514
Epoch 278
Validation binary_cross_entropy = 1.785970
Epoch 279
Loss = 3.1022e-02, PNorm = 63.2554, GNorm = 2.2324, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.809130
Epoch 280
Validation binary_cross_entropy = 1.743921
Epoch 281
Validation binary_cross_entropy = 1.652490
Epoch 282
Validation binary_cross_entropy = 1.595702
Epoch 283
Validation binary_cross_entropy = 1.574846
Epoch 284
Loss = 2.1109e-02, PNorm = 63.2854, GNorm = 1.2221, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.592967
Epoch 285
Validation binary_cross_entropy = 1.625559
Epoch 286
Validation binary_cross_entropy = 1.630575
Epoch 287
Validation binary_cross_entropy = 1.612600
Epoch 288
Validation binary_cross_entropy = 1.591713
Epoch 289
Loss = 1.4693e-02, PNorm = 63.3200, GNorm = 0.1432, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.597033
Epoch 290
Validation binary_cross_entropy = 1.599822
Epoch 291
Validation binary_cross_entropy = 1.616925
Epoch 292
Validation binary_cross_entropy = 1.634283
Epoch 293
Validation binary_cross_entropy = 1.658749
Epoch 294
Loss = 2.8604e-02, PNorm = 63.3576, GNorm = 0.7052, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.655782
Epoch 295
Validation binary_cross_entropy = 1.659013
Epoch 296
Validation binary_cross_entropy = 1.650808
Epoch 297
Validation binary_cross_entropy = 1.637357
Epoch 298
Validation binary_cross_entropy = 1.633891
Epoch 299
Loss = 1.8951e-02, PNorm = 63.4023, GNorm = 0.2482, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.582583
Model 0 best validation binary_cross_entropy = 0.343504 on epoch 31
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.263363
Ensemble test binary_cross_entropy = 0.263363
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.521846
Epoch 1
Validation binary_cross_entropy = 0.465060
Epoch 2
Validation binary_cross_entropy = 0.945279
Epoch 3
Validation binary_cross_entropy = 1.168072
Epoch 4
Loss = 1.5944e+00, PNorm = 54.1597, GNorm = 18.8047, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.764596
Epoch 5
Validation binary_cross_entropy = 0.494527
Epoch 6
Validation binary_cross_entropy = 0.437975
Epoch 7
Validation binary_cross_entropy = 0.541482
Epoch 8
Validation binary_cross_entropy = 1.074575
Epoch 9
Loss = 1.3173e+00, PNorm = 54.3334, GNorm = 38.6947, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.707493
Epoch 10
Validation binary_cross_entropy = 0.501266
Epoch 11
Validation binary_cross_entropy = 0.477301
Epoch 12
Validation binary_cross_entropy = 0.501149
Epoch 13
Validation binary_cross_entropy = 0.575067
Epoch 14
Loss = 1.5740e+00, PNorm = 54.4892, GNorm = 12.5302, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.663297
Epoch 15
Validation binary_cross_entropy = 0.685602
Epoch 16
Validation binary_cross_entropy = 0.813198
Epoch 17
Validation binary_cross_entropy = 0.702426
Epoch 18
Validation binary_cross_entropy = 0.483321
Epoch 19
Loss = 1.2309e+00, PNorm = 54.6016, GNorm = 11.9238, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.444235
Epoch 20
Validation binary_cross_entropy = 0.446018
Epoch 21
Validation binary_cross_entropy = 0.485063
Epoch 22
Validation binary_cross_entropy = 0.417255
Epoch 23
Validation binary_cross_entropy = 0.403130
Epoch 24
Loss = 9.5975e-01, PNorm = 54.7065, GNorm = 8.4359, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.451328
Epoch 25
Validation binary_cross_entropy = 0.590001
Epoch 26
Validation binary_cross_entropy = 0.678544
Epoch 27
Validation binary_cross_entropy = 0.417830
Epoch 28
Validation binary_cross_entropy = 0.350103
Epoch 29
Loss = 1.0562e+00, PNorm = 54.8249, GNorm = 10.2150, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.705402
Epoch 30
Validation binary_cross_entropy = 0.787363
Epoch 31
Validation binary_cross_entropy = 0.453873
Epoch 32
Validation binary_cross_entropy = 0.381915
Epoch 33
Validation binary_cross_entropy = 0.384671
Epoch 34
Loss = 8.6212e-01, PNorm = 54.9402, GNorm = 15.5254, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.461407
Epoch 35
Validation binary_cross_entropy = 0.577198
Epoch 36
Validation binary_cross_entropy = 0.549098
Epoch 37
Validation binary_cross_entropy = 0.465218
Epoch 38
Validation binary_cross_entropy = 0.475148
Epoch 39
Loss = 5.9019e-01, PNorm = 55.0743, GNorm = 10.4174, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.395626
Epoch 40
Validation binary_cross_entropy = 0.359914
Epoch 41
Validation binary_cross_entropy = 0.410065
Epoch 42
Validation binary_cross_entropy = 0.434705
Epoch 43
Validation binary_cross_entropy = 0.400883
Epoch 44
Loss = 8.2033e-01, PNorm = 55.1856, GNorm = 25.2617, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.386740
Epoch 45
Validation binary_cross_entropy = 0.358189
Epoch 46
Validation binary_cross_entropy = 0.509020
Epoch 47
Validation binary_cross_entropy = 0.574832
Epoch 48
Validation binary_cross_entropy = 0.402070
Epoch 49
Loss = 9.2300e-01, PNorm = 55.3391, GNorm = 11.1417, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.356915
Epoch 50
Validation binary_cross_entropy = 0.359760
Epoch 51
Validation binary_cross_entropy = 0.426367
Epoch 52
Validation binary_cross_entropy = 0.408429
Epoch 53
Validation binary_cross_entropy = 0.379359
Epoch 54
Loss = 7.0805e-01, PNorm = 55.5059, GNorm = 19.9777, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.373259
Epoch 55
Validation binary_cross_entropy = 0.483038
Epoch 56
Validation binary_cross_entropy = 0.410905
Epoch 57
Validation binary_cross_entropy = 0.396065
Epoch 58
Validation binary_cross_entropy = 0.657165
Epoch 59
Loss = 5.9330e-01, PNorm = 55.6976, GNorm = 19.6835, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.362301
Epoch 60
Validation binary_cross_entropy = 0.418914
Epoch 61
Validation binary_cross_entropy = 0.915681
Epoch 62
Validation binary_cross_entropy = 0.460524
Epoch 63
Validation binary_cross_entropy = 0.356872
Epoch 64
Loss = 9.3820e-01, PNorm = 55.8954, GNorm = 14.2106, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.356856
Epoch 65
Validation binary_cross_entropy = 0.489662
Epoch 66
Validation binary_cross_entropy = 0.647873
Epoch 67
Validation binary_cross_entropy = 0.409446
Epoch 68
Validation binary_cross_entropy = 0.391365
Epoch 69
Loss = 6.8921e-01, PNorm = 56.1155, GNorm = 18.6268, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.483329
Epoch 70
Validation binary_cross_entropy = 0.522548
Epoch 71
Validation binary_cross_entropy = 0.497314
Epoch 72
Validation binary_cross_entropy = 0.402001
Epoch 73
Validation binary_cross_entropy = 0.422464
Epoch 74
Loss = 6.4135e-01, PNorm = 56.3455, GNorm = 28.1286, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.447355
Epoch 75
Validation binary_cross_entropy = 0.521395
Epoch 76
Validation binary_cross_entropy = 0.395537
Epoch 77
Validation binary_cross_entropy = 0.392596
Epoch 78
Validation binary_cross_entropy = 0.471016
Epoch 79
Loss = 5.2074e-01, PNorm = 56.5906, GNorm = 25.2093, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.505513
Epoch 80
Validation binary_cross_entropy = 0.398192
Epoch 81
Validation binary_cross_entropy = 0.389496
Epoch 82
Validation binary_cross_entropy = 0.413213
Epoch 83
Validation binary_cross_entropy = 0.608353
Epoch 84
Loss = 5.0128e-01, PNorm = 56.8217, GNorm = 25.5405, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.421920
Epoch 85
Validation binary_cross_entropy = 0.430600
Epoch 86
Validation binary_cross_entropy = 0.426241
Epoch 87
Validation binary_cross_entropy = 0.458266
Epoch 88
Validation binary_cross_entropy = 0.494943
Epoch 89
Loss = 6.5427e-01, PNorm = 57.0781, GNorm = 9.1030, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.532622
Epoch 90
Validation binary_cross_entropy = 0.544884
Epoch 91
Validation binary_cross_entropy = 0.604264
Epoch 92
Validation binary_cross_entropy = 0.538248
Epoch 93
Validation binary_cross_entropy = 0.962359
Epoch 94
Loss = 8.3660e-01, PNorm = 57.3922, GNorm = 61.2837, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.590602
Epoch 95
Validation binary_cross_entropy = 0.482724
Epoch 96
Validation binary_cross_entropy = 0.457911
Epoch 97
Validation binary_cross_entropy = 0.495936
Epoch 98
Validation binary_cross_entropy = 0.504577
Epoch 99
Loss = 3.4206e-01, PNorm = 57.7233, GNorm = 4.4181, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.485632
Epoch 100
Validation binary_cross_entropy = 0.515350
Epoch 101
Validation binary_cross_entropy = 0.532807
Epoch 102
Validation binary_cross_entropy = 0.604178
Epoch 103
Validation binary_cross_entropy = 0.586972
Epoch 104
Loss = 4.8136e-01, PNorm = 58.0312, GNorm = 8.7989, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.584236
Epoch 105
Validation binary_cross_entropy = 0.571033
Epoch 106
Validation binary_cross_entropy = 0.590094
Epoch 107
Validation binary_cross_entropy = 0.532739
Epoch 108
Validation binary_cross_entropy = 0.527837
Epoch 109
Loss = 1.7622e-01, PNorm = 58.3001, GNorm = 7.4620, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.614643
Epoch 110
Validation binary_cross_entropy = 0.634570
Epoch 111
Validation binary_cross_entropy = 0.563285
Epoch 112
Validation binary_cross_entropy = 0.574476
Epoch 113
Validation binary_cross_entropy = 0.614327
Epoch 114
Loss = 2.9165e-01, PNorm = 58.5798, GNorm = 18.7150, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.665944
Epoch 115
Validation binary_cross_entropy = 0.640774
Epoch 116
Validation binary_cross_entropy = 0.629938
Epoch 117
Validation binary_cross_entropy = 0.603535
Epoch 118
Validation binary_cross_entropy = 0.632032
Epoch 119
Loss = 2.4855e-01, PNorm = 58.8513, GNorm = 11.7350, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.546923
Epoch 120
Validation binary_cross_entropy = 0.553118
Epoch 121
Validation binary_cross_entropy = 0.533506
Epoch 122
Validation binary_cross_entropy = 0.557892
Epoch 123
Validation binary_cross_entropy = 0.571686
Epoch 124
Loss = 1.6230e-01, PNorm = 59.1042, GNorm = 12.2564, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.661113
Epoch 125
Validation binary_cross_entropy = 0.618239
Epoch 126
Validation binary_cross_entropy = 0.786738
Epoch 127
Validation binary_cross_entropy = 0.673327
Epoch 128
Validation binary_cross_entropy = 0.629785
Epoch 129
Loss = 2.2161e-01, PNorm = 59.3698, GNorm = 12.0872, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.624197
Epoch 130
Validation binary_cross_entropy = 0.609930
Epoch 131
Validation binary_cross_entropy = 0.643909
Epoch 132
Validation binary_cross_entropy = 0.644396
Epoch 133
Validation binary_cross_entropy = 0.666175
Epoch 134
Loss = 1.7295e-01, PNorm = 59.6243, GNorm = 4.4659, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.703938
Epoch 135
Validation binary_cross_entropy = 0.723018
Epoch 136
Validation binary_cross_entropy = 0.740540
Epoch 137
Validation binary_cross_entropy = 0.772172
Epoch 138
Validation binary_cross_entropy = 0.798343
Epoch 139
Loss = 1.2609e-01, PNorm = 59.8299, GNorm = 4.2707, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.826615
Epoch 140
Validation binary_cross_entropy = 0.848315
Epoch 141
Validation binary_cross_entropy = 0.807190
Epoch 142
Validation binary_cross_entropy = 0.793080
Epoch 143
Validation binary_cross_entropy = 0.814162
Epoch 144
Loss = 1.1311e-01, PNorm = 59.9790, GNorm = 5.4338, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.795025
Epoch 145
Validation binary_cross_entropy = 0.792193
Epoch 146
Validation binary_cross_entropy = 0.778809
Epoch 147
Validation binary_cross_entropy = 0.821757
Epoch 148
Validation binary_cross_entropy = 0.827628
Epoch 149
Loss = 1.6804e-01, PNorm = 60.1449, GNorm = 6.7478, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.841234
Epoch 150
Validation binary_cross_entropy = 0.886517
Epoch 151
Validation binary_cross_entropy = 0.864251
Epoch 152
Validation binary_cross_entropy = 0.851098
Epoch 153
Validation binary_cross_entropy = 0.839738
Epoch 154
Loss = 6.2913e-02, PNorm = 60.2805, GNorm = 4.6156, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.828327
Epoch 155
Validation binary_cross_entropy = 0.816060
Epoch 156
Validation binary_cross_entropy = 0.825611
Epoch 157
Validation binary_cross_entropy = 0.873409
Epoch 158
Validation binary_cross_entropy = 0.964997
Epoch 159
Loss = 6.6846e-02, PNorm = 60.4190, GNorm = 2.5957, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.060999
Epoch 160
Validation binary_cross_entropy = 0.925797
Epoch 161
Validation binary_cross_entropy = 0.911642
Epoch 162
Validation binary_cross_entropy = 0.878516
Epoch 163
Validation binary_cross_entropy = 0.885796
Epoch 164
Loss = 1.4964e-01, PNorm = 60.5493, GNorm = 14.2123, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.847365
Epoch 165
Validation binary_cross_entropy = 0.850856
Epoch 166
Validation binary_cross_entropy = 1.006840
Epoch 167
Validation binary_cross_entropy = 0.838493
Epoch 168
Validation binary_cross_entropy = 0.910212
Epoch 169
Loss = 1.5253e-01, PNorm = 60.7026, GNorm = 11.1543, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.829066
Epoch 170
Validation binary_cross_entropy = 0.796499
Epoch 171
Validation binary_cross_entropy = 0.767513
Epoch 172
Validation binary_cross_entropy = 0.780124
Epoch 173
Validation binary_cross_entropy = 0.799251
Epoch 174
Loss = 2.3206e-01, PNorm = 60.8930, GNorm = 22.5281, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.832682
Epoch 175
Validation binary_cross_entropy = 0.893907
Epoch 176
Validation binary_cross_entropy = 0.870006
Epoch 177
Validation binary_cross_entropy = 0.857822
Epoch 178
Validation binary_cross_entropy = 0.875577
Epoch 179
Loss = 3.9549e-02, PNorm = 61.0422, GNorm = 2.5392, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.904827
Epoch 180
Validation binary_cross_entropy = 0.934452
Epoch 181
Validation binary_cross_entropy = 0.954395
Epoch 182
Validation binary_cross_entropy = 0.967315
Epoch 183
Validation binary_cross_entropy = 0.970736
Epoch 184
Loss = 2.8127e-02, PNorm = 61.1612, GNorm = 3.9425, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.987695
Epoch 185
Validation binary_cross_entropy = 1.027547
Epoch 186
Validation binary_cross_entropy = 1.073795
Epoch 187
Validation binary_cross_entropy = 1.142492
Epoch 188
Validation binary_cross_entropy = 1.117902
Epoch 189
Loss = 4.3032e-02, PNorm = 61.2522, GNorm = 1.1301, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.017053
Epoch 190
Validation binary_cross_entropy = 0.987620
Epoch 191
Validation binary_cross_entropy = 0.962486
Epoch 192
Validation binary_cross_entropy = 1.022711
Epoch 193
Validation binary_cross_entropy = 0.992589
Epoch 194
Loss = 1.7173e-02, PNorm = 61.3449, GNorm = 2.5731, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.955151
Epoch 195
Validation binary_cross_entropy = 0.949454
Epoch 196
Validation binary_cross_entropy = 1.001278
Epoch 197
Validation binary_cross_entropy = 1.072348
Epoch 198
Validation binary_cross_entropy = 0.984950
Epoch 199
Loss = 2.0712e-02, PNorm = 61.4390, GNorm = 1.2802, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.912896
Epoch 200
Validation binary_cross_entropy = 0.894520
Epoch 201
Validation binary_cross_entropy = 0.915483
Epoch 202
Validation binary_cross_entropy = 0.973758
Epoch 203
Validation binary_cross_entropy = 1.012265
Epoch 204
Loss = 3.5154e-02, PNorm = 61.5536, GNorm = 4.4396, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.974580
Epoch 205
Validation binary_cross_entropy = 0.941002
Epoch 206
Validation binary_cross_entropy = 1.001972
Epoch 207
Validation binary_cross_entropy = 0.993436
Epoch 208
Validation binary_cross_entropy = 1.044307
Epoch 209
Loss = 7.1713e-02, PNorm = 61.6874, GNorm = 2.7231, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.188561
Epoch 210
Validation binary_cross_entropy = 1.053484
Epoch 211
Validation binary_cross_entropy = 0.990284
Epoch 212
Validation binary_cross_entropy = 0.921031
Epoch 213
Validation binary_cross_entropy = 0.852072
Epoch 214
Loss = 6.2879e-02, PNorm = 61.8582, GNorm = 3.2741, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.883566
Epoch 215
Validation binary_cross_entropy = 0.899661
Epoch 216
Validation binary_cross_entropy = 0.871453
Epoch 217
Validation binary_cross_entropy = 0.872066
Epoch 218
Validation binary_cross_entropy = 0.899047
Epoch 219
Loss = 2.0697e-02, PNorm = 62.0286, GNorm = 0.6804, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.941653
Epoch 220
Validation binary_cross_entropy = 1.003692
Epoch 221
Validation binary_cross_entropy = 1.061233
Epoch 222
Validation binary_cross_entropy = 1.086453
Epoch 223
Validation binary_cross_entropy = 1.067896
Epoch 224
Loss = 6.1466e-03, PNorm = 62.1417, GNorm = 1.0557, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.040136
Epoch 225
Validation binary_cross_entropy = 1.037484
Epoch 226
Validation binary_cross_entropy = 1.053152
Epoch 227
Validation binary_cross_entropy = 1.095889
Epoch 228
Validation binary_cross_entropy = 1.110418
Epoch 229
Loss = 4.7692e-02, PNorm = 62.2147, GNorm = 4.4975, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.106863
Epoch 230
Validation binary_cross_entropy = 1.081982
Epoch 231
Validation binary_cross_entropy = 1.064359
Epoch 232
Validation binary_cross_entropy = 1.086056
Epoch 233
Validation binary_cross_entropy = 1.101550
Epoch 234
Loss = 2.5977e-02, PNorm = 62.2847, GNorm = 1.7213, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.097781
Epoch 235
Validation binary_cross_entropy = 1.105620
Epoch 236
Validation binary_cross_entropy = 1.099535
Epoch 237
Validation binary_cross_entropy = 1.101171
Epoch 238
Validation binary_cross_entropy = 1.153615
Epoch 239
Loss = 1.2833e-02, PNorm = 62.3784, GNorm = 1.3334, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.252048
Epoch 240
Validation binary_cross_entropy = 1.243075
Epoch 241
Validation binary_cross_entropy = 1.144500
Epoch 242
Validation binary_cross_entropy = 1.108733
Epoch 243
Validation binary_cross_entropy = 1.108077
Epoch 244
Loss = 2.8941e-02, PNorm = 62.4459, GNorm = 7.4401, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.110161
Epoch 245
Validation binary_cross_entropy = 1.123661
Epoch 246
Validation binary_cross_entropy = 1.154490
Epoch 247
Validation binary_cross_entropy = 1.197508
Epoch 248
Validation binary_cross_entropy = 1.223140
Epoch 249
Loss = 4.6449e-03, PNorm = 62.5068, GNorm = 1.1356, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.235656
Epoch 250
Validation binary_cross_entropy = 1.281744
Epoch 251
Validation binary_cross_entropy = 1.307018
Epoch 252
Validation binary_cross_entropy = 1.347271
Epoch 253
Validation binary_cross_entropy = 1.402199
Epoch 254
Loss = 2.1493e-02, PNorm = 62.5859, GNorm = 0.5933, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.507262
Epoch 255
Validation binary_cross_entropy = 1.585201
Epoch 256
Validation binary_cross_entropy = 1.671494
Epoch 257
Validation binary_cross_entropy = 1.616735
Epoch 258
Validation binary_cross_entropy = 1.633802
Epoch 259
Loss = 8.2407e-03, PNorm = 62.6773, GNorm = 0.7676, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.659911
Epoch 260
Validation binary_cross_entropy = 1.576661
Epoch 261
Validation binary_cross_entropy = 1.467769
Epoch 262
Validation binary_cross_entropy = 1.384441
Epoch 263
Validation binary_cross_entropy = 1.324744
Epoch 264
Loss = 7.5974e-03, PNorm = 62.7353, GNorm = 0.4136, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.281074
Epoch 265
Validation binary_cross_entropy = 1.260015
Epoch 266
Validation binary_cross_entropy = 1.277297
Epoch 267
Validation binary_cross_entropy = 1.286946
Epoch 268
Validation binary_cross_entropy = 1.324681
Epoch 269
Loss = 3.9393e-02, PNorm = 62.8125, GNorm = 1.8558, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.349791
Epoch 270
Validation binary_cross_entropy = 1.351031
Epoch 271
Validation binary_cross_entropy = 1.382756
Epoch 272
Validation binary_cross_entropy = 1.423898
Epoch 273
Validation binary_cross_entropy = 1.466697
Epoch 274
Loss = 3.4575e-03, PNorm = 62.8979, GNorm = 0.7192, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.515767
Epoch 275
Validation binary_cross_entropy = 1.593350
Epoch 276
Validation binary_cross_entropy = 1.554917
Epoch 277
Validation binary_cross_entropy = 1.354958
Epoch 278
Validation binary_cross_entropy = 1.233689
Epoch 279
Loss = 9.6611e-03, PNorm = 62.9920, GNorm = 1.2381, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.187982
Epoch 280
Validation binary_cross_entropy = 1.175531
Epoch 281
Validation binary_cross_entropy = 1.184141
Epoch 282
Validation binary_cross_entropy = 1.223588
Epoch 283
Validation binary_cross_entropy = 1.269700
Epoch 284
Loss = 3.0618e-02, PNorm = 63.0827, GNorm = 0.3961, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.342752
Epoch 285
Validation binary_cross_entropy = 1.379976
Epoch 286
Validation binary_cross_entropy = 1.432435
Epoch 287
Validation binary_cross_entropy = 1.464759
Epoch 288
Validation binary_cross_entropy = 1.480862
Epoch 289
Loss = 2.9200e-03, PNorm = 63.1502, GNorm = 0.2418, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.510083
Epoch 290
Validation binary_cross_entropy = 1.576274
Epoch 291
Validation binary_cross_entropy = 1.652131
Epoch 292
Validation binary_cross_entropy = 1.609407
Epoch 293
Validation binary_cross_entropy = 1.573870
Epoch 294
Loss = 1.5401e-03, PNorm = 63.2047, GNorm = 0.0427, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.554498
Epoch 295
Validation binary_cross_entropy = 1.560272
Epoch 296
Validation binary_cross_entropy = 1.570652
Epoch 297
Validation binary_cross_entropy = 1.588282
Epoch 298
Validation binary_cross_entropy = 1.631740
Epoch 299
Loss = 8.4950e-03, PNorm = 63.2406, GNorm = 1.8559, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.656323
Model 0 best validation binary_cross_entropy = 0.350103 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.250060
Ensemble test binary_cross_entropy = 0.250060
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.501388
Epoch 1
Validation binary_cross_entropy = 0.470962
Epoch 2
Validation binary_cross_entropy = 0.703400
Epoch 3
Validation binary_cross_entropy = 0.874536
Epoch 4
Loss = 1.4734e+00, PNorm = 54.1612, GNorm = 20.3956, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.779067
Epoch 5
Validation binary_cross_entropy = 0.556787
Epoch 6
Validation binary_cross_entropy = 0.494719
Epoch 7
Validation binary_cross_entropy = 0.666001
Epoch 8
Validation binary_cross_entropy = 0.610363
Epoch 9
Loss = 1.5301e+00, PNorm = 54.3462, GNorm = 22.3216, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.683196
Epoch 10
Validation binary_cross_entropy = 0.793379
Epoch 11
Validation binary_cross_entropy = 0.594466
Epoch 12
Validation binary_cross_entropy = 0.690960
Epoch 13
Validation binary_cross_entropy = 0.758522
Epoch 14
Loss = 1.2694e+00, PNorm = 54.5298, GNorm = 24.4859, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.472952
Epoch 15
Validation binary_cross_entropy = 0.426295
Epoch 16
Validation binary_cross_entropy = 0.441637
Epoch 17
Validation binary_cross_entropy = 0.531751
Epoch 18
Validation binary_cross_entropy = 0.562179
Epoch 19
Loss = 1.2096e+00, PNorm = 54.6678, GNorm = 9.9858, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.558356
Epoch 20
Validation binary_cross_entropy = 0.460106
Epoch 21
Validation binary_cross_entropy = 0.461624
Epoch 22
Validation binary_cross_entropy = 0.399045
Epoch 23
Validation binary_cross_entropy = 0.380048
Epoch 24
Loss = 1.1275e+00, PNorm = 54.8010, GNorm = 10.9741, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.443549
Epoch 25
Validation binary_cross_entropy = 0.472154
Epoch 26
Validation binary_cross_entropy = 0.512972
Epoch 27
Validation binary_cross_entropy = 0.493990
Epoch 28
Validation binary_cross_entropy = 0.395328
Epoch 29
Loss = 1.0963e+00, PNorm = 54.9345, GNorm = 24.4610, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.380759
Epoch 30
Validation binary_cross_entropy = 0.586129
Epoch 31
Validation binary_cross_entropy = 0.592930
Epoch 32
Validation binary_cross_entropy = 0.364563
Epoch 33
Validation binary_cross_entropy = 0.489011
Epoch 34
Loss = 1.0983e+00, PNorm = 55.0856, GNorm = 32.5304, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.393787
Epoch 35
Validation binary_cross_entropy = 0.366124
Epoch 36
Validation binary_cross_entropy = 0.391085
Epoch 37
Validation binary_cross_entropy = 0.472984
Epoch 38
Validation binary_cross_entropy = 0.417431
Epoch 39
Loss = 8.7050e-01, PNorm = 55.2345, GNorm = 10.5511, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.374743
Epoch 40
Validation binary_cross_entropy = 0.406153
Epoch 41
Validation binary_cross_entropy = 0.531418
Epoch 42
Validation binary_cross_entropy = 0.433157
Epoch 43
Validation binary_cross_entropy = 0.435680
Epoch 44
Loss = 7.4181e-01, PNorm = 55.4286, GNorm = 21.2052, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.732759
Epoch 45
Validation binary_cross_entropy = 0.371149
Epoch 46
Validation binary_cross_entropy = 0.359082
Epoch 47
Validation binary_cross_entropy = 0.412079
Epoch 48
Validation binary_cross_entropy = 0.438119
Epoch 49
Loss = 9.3257e-01, PNorm = 55.6063, GNorm = 21.8839, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.397874
Epoch 50
Validation binary_cross_entropy = 0.367388
Epoch 51
Validation binary_cross_entropy = 0.397217
Epoch 52
Validation binary_cross_entropy = 0.450203
Epoch 53
Validation binary_cross_entropy = 0.371249
Epoch 54
Loss = 5.7028e-01, PNorm = 55.8177, GNorm = 6.6252, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.366547
Epoch 55
Validation binary_cross_entropy = 0.385540
Epoch 56
Validation binary_cross_entropy = 0.426605
Epoch 57
Validation binary_cross_entropy = 0.437554
Epoch 58
Validation binary_cross_entropy = 0.516123
Epoch 59
Loss = 5.0457e-01, PNorm = 55.9868, GNorm = 20.8623, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.457739
Epoch 60
Validation binary_cross_entropy = 0.434984
Epoch 61
Validation binary_cross_entropy = 0.759101
Epoch 62
Validation binary_cross_entropy = 0.665091
Epoch 63
Validation binary_cross_entropy = 0.456567
Epoch 64
Loss = 5.8286e-01, PNorm = 56.1918, GNorm = 15.2040, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.659175
Epoch 65
Validation binary_cross_entropy = 0.851901
Epoch 66
Validation binary_cross_entropy = 0.415389
Epoch 67
Validation binary_cross_entropy = 0.440459
Epoch 68
Validation binary_cross_entropy = 0.424582
Epoch 69
Loss = 3.8483e-01, PNorm = 56.4194, GNorm = 18.6827, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.924526
Epoch 70
Validation binary_cross_entropy = 0.540200
Epoch 71
Validation binary_cross_entropy = 0.392700
Epoch 72
Validation binary_cross_entropy = 0.392667
Epoch 73
Validation binary_cross_entropy = 0.428107
Epoch 74
Loss = 2.7734e-01, PNorm = 56.6879, GNorm = 6.0539, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.616885
Epoch 75
Validation binary_cross_entropy = 0.508804
Epoch 76
Validation binary_cross_entropy = 0.420997
Epoch 77
Validation binary_cross_entropy = 0.428077
Epoch 78
Validation binary_cross_entropy = 0.416196
Epoch 79
Loss = 4.3558e-01, PNorm = 56.9183, GNorm = 9.5456, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.534778
Epoch 80
Validation binary_cross_entropy = 0.505023
Epoch 81
Validation binary_cross_entropy = 0.426177
Epoch 82
Validation binary_cross_entropy = 0.429148
Epoch 83
Validation binary_cross_entropy = 0.521261
Epoch 84
Loss = 7.8839e-01, PNorm = 57.1454, GNorm = 42.2771, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.578378
Epoch 85
Validation binary_cross_entropy = 0.464423
Epoch 86
Validation binary_cross_entropy = 0.484990
Epoch 87
Validation binary_cross_entropy = 0.665867
Epoch 88
Validation binary_cross_entropy = 0.649816
Epoch 89
Loss = 3.0589e-01, PNorm = 57.4244, GNorm = 6.2445, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.527654
Epoch 90
Validation binary_cross_entropy = 0.517920
Epoch 91
Validation binary_cross_entropy = 0.506608
Epoch 92
Validation binary_cross_entropy = 0.502322
Epoch 93
Validation binary_cross_entropy = 0.547251
Epoch 94
Loss = 4.4489e-01, PNorm = 57.7013, GNorm = 15.6901, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.507197
Epoch 95
Validation binary_cross_entropy = 0.504279
Epoch 96
Validation binary_cross_entropy = 0.510709
Epoch 97
Validation binary_cross_entropy = 0.528765
Epoch 98
Validation binary_cross_entropy = 0.621913
Epoch 99
Loss = 3.4609e-01, PNorm = 57.9673, GNorm = 13.5819, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.634893
Epoch 100
Validation binary_cross_entropy = 0.592257
Epoch 101
Validation binary_cross_entropy = 0.606351
Epoch 102
Validation binary_cross_entropy = 0.674317
Epoch 103
Validation binary_cross_entropy = 0.547684
Epoch 104
Loss = 1.6952e-01, PNorm = 58.2382, GNorm = 5.8024, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.579872
Epoch 105
Validation binary_cross_entropy = 0.630683
Epoch 106
Validation binary_cross_entropy = 0.605939
Epoch 107
Validation binary_cross_entropy = 0.649205
Epoch 108
Validation binary_cross_entropy = 0.636442
Epoch 109
Loss = 1.4934e-01, PNorm = 58.5210, GNorm = 3.6882, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.606542
Epoch 110
Validation binary_cross_entropy = 0.638209
Epoch 111
Validation binary_cross_entropy = 0.629575
Epoch 112
Validation binary_cross_entropy = 0.669843
Epoch 113
Validation binary_cross_entropy = 1.054108
Epoch 114
Loss = 3.7896e-01, PNorm = 58.8176, GNorm = 20.0602, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.704725
Epoch 115
Validation binary_cross_entropy = 0.664631
Epoch 116
Validation binary_cross_entropy = 0.619304
Epoch 117
Validation binary_cross_entropy = 0.680834
Epoch 118
Validation binary_cross_entropy = 0.631503
Epoch 119
Loss = 2.2698e-01, PNorm = 59.1495, GNorm = 9.4476, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.665127
Epoch 120
Validation binary_cross_entropy = 0.741372
Epoch 121
Validation binary_cross_entropy = 0.780059
Epoch 122
Validation binary_cross_entropy = 0.743533
Epoch 123
Validation binary_cross_entropy = 0.705713
Epoch 124
Loss = 1.1877e-01, PNorm = 59.4389, GNorm = 7.3004, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.728419
Epoch 125
Validation binary_cross_entropy = 0.827024
Epoch 126
Validation binary_cross_entropy = 0.795205
Epoch 127
Validation binary_cross_entropy = 0.848418
Epoch 128
Validation binary_cross_entropy = 0.824376
Epoch 129
Loss = 2.0224e-01, PNorm = 59.6774, GNorm = 3.4102, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.764329
Epoch 130
Validation binary_cross_entropy = 0.704533
Epoch 131
Validation binary_cross_entropy = 0.666489
Epoch 132
Validation binary_cross_entropy = 0.644230
Epoch 133
Validation binary_cross_entropy = 0.631392
Epoch 134
Loss = 1.0207e-01, PNorm = 59.8717, GNorm = 2.5113, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.674180
Epoch 135
Validation binary_cross_entropy = 0.734073
Epoch 136
Validation binary_cross_entropy = 0.705350
Epoch 137
Validation binary_cross_entropy = 0.741076
Epoch 138
Validation binary_cross_entropy = 0.790729
Epoch 139
Loss = 2.0071e-01, PNorm = 60.0589, GNorm = 8.8717, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.841918
Epoch 140
Validation binary_cross_entropy = 0.876090
Epoch 141
Validation binary_cross_entropy = 0.915882
Epoch 142
Validation binary_cross_entropy = 0.888634
Epoch 143
Validation binary_cross_entropy = 0.894859
Epoch 144
Loss = 5.0637e-02, PNorm = 60.2517, GNorm = 3.3134, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.894063
Epoch 145
Validation binary_cross_entropy = 0.868815
Epoch 146
Validation binary_cross_entropy = 0.878237
Epoch 147
Validation binary_cross_entropy = 0.841242
Epoch 148
Validation binary_cross_entropy = 0.834614
Epoch 149
Loss = 1.1896e-01, PNorm = 60.4306, GNorm = 5.7612, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.845693
Epoch 150
Validation binary_cross_entropy = 0.877593
Epoch 151
Validation binary_cross_entropy = 0.928281
Epoch 152
Validation binary_cross_entropy = 0.991308
Epoch 153
Validation binary_cross_entropy = 0.940296
Epoch 154
Loss = 1.7388e-01, PNorm = 60.6137, GNorm = 11.8701, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.920521
Epoch 155
Validation binary_cross_entropy = 0.959588
Epoch 156
Validation binary_cross_entropy = 0.975671
Epoch 157
Validation binary_cross_entropy = 0.981769
Epoch 158
Validation binary_cross_entropy = 0.966651
Epoch 159
Loss = 1.0216e-01, PNorm = 60.7504, GNorm = 5.8416, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.897865
Epoch 160
Validation binary_cross_entropy = 0.890442
Epoch 161
Validation binary_cross_entropy = 0.865806
Epoch 162
Validation binary_cross_entropy = 0.898768
Epoch 163
Validation binary_cross_entropy = 0.884494
Epoch 164
Loss = 1.9936e-01, PNorm = 60.9066, GNorm = 6.0648, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.903999
Epoch 165
Validation binary_cross_entropy = 0.958029
Epoch 166
Validation binary_cross_entropy = 0.945767
Epoch 167
Validation binary_cross_entropy = 1.073373
Epoch 168
Validation binary_cross_entropy = 1.304558
Epoch 169
Loss = 2.8315e-01, PNorm = 61.0805, GNorm = 23.6607, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.023681
Epoch 170
Validation binary_cross_entropy = 0.930945
Epoch 171
Validation binary_cross_entropy = 0.859330
Epoch 172
Validation binary_cross_entropy = 0.810268
Epoch 173
Validation binary_cross_entropy = 0.784805
Epoch 174
Loss = 1.1436e-01, PNorm = 61.2539, GNorm = 11.5447, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.731619
Epoch 175
Validation binary_cross_entropy = 0.746706
Epoch 176
Validation binary_cross_entropy = 0.765949
Epoch 177
Validation binary_cross_entropy = 0.791854
Epoch 178
Validation binary_cross_entropy = 0.840794
Epoch 179
Loss = 4.3316e-02, PNorm = 61.4089, GNorm = 0.5190, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.880804
Epoch 180
Validation binary_cross_entropy = 0.912354
Epoch 181
Validation binary_cross_entropy = 0.936230
Epoch 182
Validation binary_cross_entropy = 0.965357
Epoch 183
Validation binary_cross_entropy = 0.957559
Epoch 184
Loss = 5.9626e-02, PNorm = 61.5292, GNorm = 2.9373, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.938127
Epoch 185
Validation binary_cross_entropy = 0.952712
Epoch 186
Validation binary_cross_entropy = 0.970622
Epoch 187
Validation binary_cross_entropy = 0.999769
Epoch 188
Validation binary_cross_entropy = 1.020011
Epoch 189
Loss = 7.5778e-02, PNorm = 61.6470, GNorm = 7.1966, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.993577
Epoch 190
Validation binary_cross_entropy = 0.990557
Epoch 191
Validation binary_cross_entropy = 1.074455
Epoch 192
Validation binary_cross_entropy = 1.081700
Epoch 193
Validation binary_cross_entropy = 0.992528
Epoch 194
Loss = 1.5675e-01, PNorm = 61.7566, GNorm = 4.4316, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.957966
Epoch 195
Validation binary_cross_entropy = 0.947478
Epoch 196
Validation binary_cross_entropy = 0.961775
Epoch 197
Validation binary_cross_entropy = 0.968548
Epoch 198
Validation binary_cross_entropy = 0.975502
Epoch 199
Loss = 6.4655e-03, PNorm = 61.8745, GNorm = 1.4919, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.983370
Epoch 200
Validation binary_cross_entropy = 1.006825
Epoch 201
Validation binary_cross_entropy = 0.998775
Epoch 202
Validation binary_cross_entropy = 1.023599
Epoch 203
Validation binary_cross_entropy = 1.030751
Epoch 204
Loss = 4.4748e-02, PNorm = 61.9846, GNorm = 5.3019, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.063399
Epoch 205
Validation binary_cross_entropy = 1.112369
Epoch 206
Validation binary_cross_entropy = 1.698792
Epoch 207
Validation binary_cross_entropy = 1.259854
Epoch 208
Validation binary_cross_entropy = 1.212230
Epoch 209
Loss = 3.7971e-02, PNorm = 62.1056, GNorm = 1.4614, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.668589
Epoch 210
Validation binary_cross_entropy = 1.655711
Epoch 211
Validation binary_cross_entropy = 1.124043
Epoch 212
Validation binary_cross_entropy = 1.127516
Epoch 213
Validation binary_cross_entropy = 1.117703
Epoch 214
Loss = 1.1876e-01, PNorm = 62.2065, GNorm = 9.8059, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.106653
Epoch 215
Validation binary_cross_entropy = 1.091956
Epoch 216
Validation binary_cross_entropy = 1.075839
Epoch 217
Validation binary_cross_entropy = 1.054394
Epoch 218
Validation binary_cross_entropy = 1.060699
Epoch 219
Loss = 6.0274e-02, PNorm = 62.2894, GNorm = 1.9154, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.105445
Epoch 220
Validation binary_cross_entropy = 1.064804
Epoch 221
Validation binary_cross_entropy = 1.041124
Epoch 222
Validation binary_cross_entropy = 1.054319
Epoch 223
Validation binary_cross_entropy = 1.066845
Epoch 224
Loss = 1.6916e-02, PNorm = 62.3848, GNorm = 1.9461, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.075503
Epoch 225
Validation binary_cross_entropy = 1.111377
Epoch 226
Validation binary_cross_entropy = 1.145218
Epoch 227
Validation binary_cross_entropy = 1.127186
Epoch 228
Validation binary_cross_entropy = 1.169051
Epoch 229
Loss = 6.0224e-02, PNorm = 62.4679, GNorm = 1.1990, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.151438
Epoch 230
Validation binary_cross_entropy = 1.232382
Epoch 231
Validation binary_cross_entropy = 1.127272
Epoch 232
Validation binary_cross_entropy = 1.032056
Epoch 233
Validation binary_cross_entropy = 1.014078
Epoch 234
Loss = 3.5534e-02, PNorm = 62.5739, GNorm = 2.3966, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.028474
Epoch 235
Validation binary_cross_entropy = 1.130454
Epoch 236
Validation binary_cross_entropy = 1.174934
Epoch 237
Validation binary_cross_entropy = 1.132042
Epoch 238
Validation binary_cross_entropy = 1.108386
Epoch 239
Loss = 2.1557e-02, PNorm = 62.6820, GNorm = 1.1643, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.132405
Epoch 240
Validation binary_cross_entropy = 1.161309
Epoch 241
Validation binary_cross_entropy = 1.139074
Epoch 242
Validation binary_cross_entropy = 1.171083
Epoch 243
Validation binary_cross_entropy = 1.189880
Epoch 244
Loss = 1.4832e-02, PNorm = 62.7615, GNorm = 1.1101, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.182028
Epoch 245
Validation binary_cross_entropy = 1.178740
Epoch 246
Validation binary_cross_entropy = 1.181981
Epoch 247
Validation binary_cross_entropy = 1.206655
Epoch 248
Validation binary_cross_entropy = 1.226431
Epoch 249
Loss = 4.9113e-02, PNorm = 62.8329, GNorm = 8.3814, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.260073
Epoch 250
Validation binary_cross_entropy = 1.307543
Epoch 251
Validation binary_cross_entropy = 1.354890
Epoch 252
Validation binary_cross_entropy = 1.156397
Epoch 253
Validation binary_cross_entropy = 1.038328
Epoch 254
Loss = 4.1201e-02, PNorm = 62.9354, GNorm = 5.6288, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.037379
Epoch 255
Validation binary_cross_entropy = 1.010801
Epoch 256
Validation binary_cross_entropy = 1.056716
Epoch 257
Validation binary_cross_entropy = 1.087540
Epoch 258
Validation binary_cross_entropy = 1.091542
Epoch 259
Loss = 1.2056e-02, PNorm = 63.0563, GNorm = 1.1741, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.109945
Epoch 260
Validation binary_cross_entropy = 1.139266
Epoch 261
Validation binary_cross_entropy = 1.166674
Epoch 262
Validation binary_cross_entropy = 1.197386
Epoch 263
Validation binary_cross_entropy = 1.227755
Epoch 264
Loss = 1.6958e-02, PNorm = 63.1468, GNorm = 1.8442, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.251766
Epoch 265
Validation binary_cross_entropy = 1.260059
Epoch 266
Validation binary_cross_entropy = 1.250571
Epoch 267
Validation binary_cross_entropy = 1.161241
Epoch 268
Validation binary_cross_entropy = 1.138334
Epoch 269
Loss = 8.1028e-03, PNorm = 63.2235, GNorm = 0.6584, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.135005
Epoch 270
Validation binary_cross_entropy = 1.116053
Epoch 271
Validation binary_cross_entropy = 1.094395
Epoch 272
Validation binary_cross_entropy = 1.096954
Epoch 273
Validation binary_cross_entropy = 1.137722
Epoch 274
Loss = 3.5842e-02, PNorm = 63.2782, GNorm = 3.7886, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.128812
Epoch 275
Validation binary_cross_entropy = 1.123580
Epoch 276
Validation binary_cross_entropy = 1.163026
Epoch 277
Validation binary_cross_entropy = 1.208543
Epoch 278
Validation binary_cross_entropy = 1.241443
Epoch 279
Loss = 6.7792e-02, PNorm = 63.3499, GNorm = 2.4299, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.261716
Epoch 280
Validation binary_cross_entropy = 1.253785
Epoch 281
Validation binary_cross_entropy = 1.264076
Epoch 282
Validation binary_cross_entropy = 1.273788
Epoch 283
Validation binary_cross_entropy = 1.288009
Epoch 284
Loss = 1.4621e-02, PNorm = 63.4180, GNorm = 1.5197, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.295146
Epoch 285
Validation binary_cross_entropy = 1.309673
Epoch 286
Validation binary_cross_entropy = 1.307251
Epoch 287
Validation binary_cross_entropy = 1.300842
Epoch 288
Validation binary_cross_entropy = 1.304194
Epoch 289
Loss = 5.7937e-03, PNorm = 63.4654, GNorm = 2.1252, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.319181
Epoch 290
Validation binary_cross_entropy = 1.329863
Epoch 291
Validation binary_cross_entropy = 1.340725
Epoch 292
Validation binary_cross_entropy = 1.346613
Epoch 293
Validation binary_cross_entropy = 1.355434
Epoch 294
Loss = 1.8771e-02, PNorm = 63.5157, GNorm = 4.1695, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.324182
Epoch 295
Validation binary_cross_entropy = 1.296948
Epoch 296
Validation binary_cross_entropy = 1.286746
Epoch 297
Validation binary_cross_entropy = 1.286468
Epoch 298
Validation binary_cross_entropy = 1.285305
Epoch 299
Loss = 7.5217e-02, PNorm = 63.5598, GNorm = 9.4659, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.283988
Model 0 best validation binary_cross_entropy = 0.359082 on epoch 46
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.189909
Ensemble test binary_cross_entropy = 0.189909
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.522067
Epoch 1
Validation binary_cross_entropy = 0.485208
Epoch 2
Validation binary_cross_entropy = 0.757731
Epoch 3
Validation binary_cross_entropy = 0.909094
Epoch 4
Loss = 1.5233e+00, PNorm = 54.1615, GNorm = 17.4970, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.576440
Epoch 5
Validation binary_cross_entropy = 0.411006
Epoch 6
Validation binary_cross_entropy = 0.413331
Epoch 7
Validation binary_cross_entropy = 0.593428
Epoch 8
Validation binary_cross_entropy = 0.643182
Epoch 9
Loss = 1.3196e+00, PNorm = 54.3418, GNorm = 17.7285, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.680537
Epoch 10
Validation binary_cross_entropy = 0.495725
Epoch 11
Validation binary_cross_entropy = 0.462556
Epoch 12
Validation binary_cross_entropy = 0.518793
Epoch 13
Validation binary_cross_entropy = 0.752135
Epoch 14
Loss = 1.2745e+00, PNorm = 54.5194, GNorm = 12.6254, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.681127
Epoch 15
Validation binary_cross_entropy = 0.556102
Epoch 16
Validation binary_cross_entropy = 0.441859
Epoch 17
Validation binary_cross_entropy = 0.411424
Epoch 18
Validation binary_cross_entropy = 0.473132
Epoch 19
Loss = 1.3426e+00, PNorm = 54.6767, GNorm = 17.0334, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.441740
Epoch 20
Validation binary_cross_entropy = 0.451559
Epoch 21
Validation binary_cross_entropy = 0.509498
Epoch 22
Validation binary_cross_entropy = 0.424333
Epoch 23
Validation binary_cross_entropy = 0.387919
Epoch 24
Loss = 1.0749e+00, PNorm = 54.8465, GNorm = 9.5623, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.402740
Epoch 25
Validation binary_cross_entropy = 0.844350
Epoch 26
Validation binary_cross_entropy = 0.365252
Epoch 27
Validation binary_cross_entropy = 0.431795
Epoch 28
Validation binary_cross_entropy = 0.703254
Epoch 29
Loss = 7.9781e-01, PNorm = 54.9925, GNorm = 20.8322, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.349283
Epoch 30
Validation binary_cross_entropy = 0.335619
Epoch 31
Validation binary_cross_entropy = 0.492437
Epoch 32
Validation binary_cross_entropy = 0.663882
Epoch 33
Validation binary_cross_entropy = 0.561173
Epoch 34
Loss = 8.7150e-01, PNorm = 55.1298, GNorm = 15.5041, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.378595
Epoch 35
Validation binary_cross_entropy = 0.397270
Epoch 36
Validation binary_cross_entropy = 0.471759
Epoch 37
Validation binary_cross_entropy = 0.708754
Epoch 38
Validation binary_cross_entropy = 0.434824
Epoch 39
Loss = 1.1165e+00, PNorm = 55.2866, GNorm = 21.3169, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.343710
Epoch 40
Validation binary_cross_entropy = 0.524587
Epoch 41
Validation binary_cross_entropy = 0.750141
Epoch 42
Validation binary_cross_entropy = 0.412179
Epoch 43
Validation binary_cross_entropy = 0.363224
Epoch 44
Loss = 1.2948e+00, PNorm = 55.4226, GNorm = 17.9140, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.368736
Epoch 45
Validation binary_cross_entropy = 0.404446
Epoch 46
Validation binary_cross_entropy = 0.449537
Epoch 47
Validation binary_cross_entropy = 0.396699
Epoch 48
Validation binary_cross_entropy = 0.369077
Epoch 49
Loss = 6.9045e-01, PNorm = 55.6116, GNorm = 14.5687, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.660761
Epoch 50
Validation binary_cross_entropy = 0.423888
Epoch 51
Validation binary_cross_entropy = 0.380691
Epoch 52
Validation binary_cross_entropy = 0.474824
Epoch 53
Validation binary_cross_entropy = 0.455198
Epoch 54
Loss = 6.7875e-01, PNorm = 55.7888, GNorm = 9.4184, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.343767
Epoch 55
Validation binary_cross_entropy = 0.348153
Epoch 56
Validation binary_cross_entropy = 0.388083
Epoch 57
Validation binary_cross_entropy = 0.386853
Epoch 58
Validation binary_cross_entropy = 0.352793
Epoch 59
Loss = 5.5601e-01, PNorm = 56.0022, GNorm = 23.5437, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.435483
Epoch 60
Validation binary_cross_entropy = 0.406814
Epoch 61
Validation binary_cross_entropy = 0.375031
Epoch 62
Validation binary_cross_entropy = 0.491798
Epoch 63
Validation binary_cross_entropy = 0.510213
Epoch 64
Loss = 5.4297e-01, PNorm = 56.2365, GNorm = 17.7878, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.400282
Epoch 65
Validation binary_cross_entropy = 0.499149
Epoch 66
Validation binary_cross_entropy = 0.642734
Epoch 67
Validation binary_cross_entropy = 0.486353
Epoch 68
Validation binary_cross_entropy = 0.436788
Epoch 69
Loss = 7.3698e-01, PNorm = 56.5049, GNorm = 17.1536, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.485918
Epoch 70
Validation binary_cross_entropy = 0.839986
Epoch 71
Validation binary_cross_entropy = 0.447511
Epoch 72
Validation binary_cross_entropy = 0.424155
Epoch 73
Validation binary_cross_entropy = 0.536404
Epoch 74
Loss = 4.6659e-01, PNorm = 56.8028, GNorm = 15.2940, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.819522
Epoch 75
Validation binary_cross_entropy = 0.423780
Epoch 76
Validation binary_cross_entropy = 0.420720
Epoch 77
Validation binary_cross_entropy = 0.514167
Epoch 78
Validation binary_cross_entropy = 0.506315
Epoch 79
Loss = 2.9304e-01, PNorm = 57.1082, GNorm = 15.3704, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.447179
Epoch 80
Validation binary_cross_entropy = 0.518433
Epoch 81
Validation binary_cross_entropy = 0.510720
Epoch 82
Validation binary_cross_entropy = 0.517501
Epoch 83
Validation binary_cross_entropy = 0.654483
Epoch 84
Loss = 3.3714e-01, PNorm = 57.4628, GNorm = 12.2516, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.547543
Epoch 85
Validation binary_cross_entropy = 0.551570
Epoch 86
Validation binary_cross_entropy = 0.568216
Epoch 87
Validation binary_cross_entropy = 0.671679
Epoch 88
Validation binary_cross_entropy = 0.603717
Epoch 89
Loss = 3.1453e-01, PNorm = 57.7784, GNorm = 14.1034, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.569400
Epoch 90
Validation binary_cross_entropy = 0.574996
Epoch 91
Validation binary_cross_entropy = 0.589236
Epoch 92
Validation binary_cross_entropy = 0.575470
Epoch 93
Validation binary_cross_entropy = 0.785260
Epoch 94
Loss = 3.5858e-01, PNorm = 58.0939, GNorm = 14.3403, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.653344
Epoch 95
Validation binary_cross_entropy = 0.665598
Epoch 96
Validation binary_cross_entropy = 0.714399
Epoch 97
Validation binary_cross_entropy = 0.612528
Epoch 98
Validation binary_cross_entropy = 0.576085
Epoch 99
Loss = 3.8696e-01, PNorm = 58.3983, GNorm = 36.3225, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.598311
Epoch 100
Validation binary_cross_entropy = 0.707551
Epoch 101
Validation binary_cross_entropy = 0.716907
Epoch 102
Validation binary_cross_entropy = 0.674919
Epoch 103
Validation binary_cross_entropy = 0.712762
Epoch 104
Loss = 2.1927e-01, PNorm = 58.6782, GNorm = 7.3030, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.732543
Epoch 105
Validation binary_cross_entropy = 0.760347
Epoch 106
Validation binary_cross_entropy = 0.788892
Epoch 107
Validation binary_cross_entropy = 0.724505
Epoch 108
Validation binary_cross_entropy = 0.639370
Epoch 109
Loss = 1.5764e-01, PNorm = 58.9233, GNorm = 4.3721, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.629959
Epoch 110
Validation binary_cross_entropy = 0.572749
Epoch 111
Validation binary_cross_entropy = 0.569817
Epoch 112
Validation binary_cross_entropy = 0.589137
Epoch 113
Validation binary_cross_entropy = 0.649257
Epoch 114
Loss = 2.0333e-01, PNorm = 59.1647, GNorm = 10.5787, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.604488
Epoch 115
Validation binary_cross_entropy = 0.632488
Epoch 116
Validation binary_cross_entropy = 0.655598
Epoch 117
Validation binary_cross_entropy = 0.741249
Epoch 118
Validation binary_cross_entropy = 0.746355
Epoch 119
Loss = 1.0258e-01, PNorm = 59.3887, GNorm = 13.5915, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.799277
Epoch 120
Validation binary_cross_entropy = 0.872726
Epoch 121
Validation binary_cross_entropy = 0.724853
Epoch 122
Validation binary_cross_entropy = 0.769245
Epoch 123
Validation binary_cross_entropy = 0.681560
Epoch 124
Loss = 1.2378e-01, PNorm = 59.5856, GNorm = 4.6941, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.622862
Epoch 125
Validation binary_cross_entropy = 0.635546
Epoch 126
Validation binary_cross_entropy = 0.664587
Epoch 127
Validation binary_cross_entropy = 0.701839
Epoch 128
Validation binary_cross_entropy = 0.829260
Epoch 129
Loss = 1.1589e-01, PNorm = 59.7643, GNorm = 2.2306, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.791987
Epoch 130
Validation binary_cross_entropy = 0.702527
Epoch 131
Validation binary_cross_entropy = 0.691728
Epoch 132
Validation binary_cross_entropy = 0.762320
Epoch 133
Validation binary_cross_entropy = 0.748346
Epoch 134
Loss = 8.8487e-02, PNorm = 59.9357, GNorm = 8.7781, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.781283
Epoch 135
Validation binary_cross_entropy = 0.856066
Epoch 136
Validation binary_cross_entropy = 0.875871
Epoch 137
Validation binary_cross_entropy = 0.856357
Epoch 138
Validation binary_cross_entropy = 0.858634
Epoch 139
Loss = 9.6571e-02, PNorm = 60.1047, GNorm = 2.1919, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.878762
Epoch 140
Validation binary_cross_entropy = 0.852500
Epoch 141
Validation binary_cross_entropy = 0.839594
Epoch 142
Validation binary_cross_entropy = 0.969229
Epoch 143
Validation binary_cross_entropy = 0.912652
Epoch 144
Loss = 8.0742e-02, PNorm = 60.2760, GNorm = 4.3201, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.928631
Epoch 145
Validation binary_cross_entropy = 0.994907
Epoch 146
Validation binary_cross_entropy = 1.074451
Epoch 147
Validation binary_cross_entropy = 0.930182
Epoch 148
Validation binary_cross_entropy = 0.896522
Epoch 149
Loss = 7.2347e-02, PNorm = 60.4373, GNorm = 3.8692, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.795532
Epoch 150
Validation binary_cross_entropy = 0.749149
Epoch 151
Validation binary_cross_entropy = 0.741103
Epoch 152
Validation binary_cross_entropy = 0.792694
Epoch 153
Validation binary_cross_entropy = 0.878482
Epoch 154
Loss = 1.2321e-01, PNorm = 60.5917, GNorm = 3.5717, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.947311
Epoch 155
Validation binary_cross_entropy = 0.970030
Epoch 156
Validation binary_cross_entropy = 0.953879
Epoch 157
Validation binary_cross_entropy = 0.951884
Epoch 158
Validation binary_cross_entropy = 1.144960
Epoch 159
Loss = 2.0785e-01, PNorm = 60.7409, GNorm = 9.4915, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.929573
Epoch 160
Validation binary_cross_entropy = 0.928786
Epoch 161
Validation binary_cross_entropy = 0.908293
Epoch 162
Validation binary_cross_entropy = 0.878906
Epoch 163
Validation binary_cross_entropy = 0.826869
Epoch 164
Loss = 1.2838e-01, PNorm = 60.9229, GNorm = 2.2812, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.815784
Epoch 165
Validation binary_cross_entropy = 0.798151
Epoch 166
Validation binary_cross_entropy = 0.826939
Epoch 167
Validation binary_cross_entropy = 0.811409
Epoch 168
Validation binary_cross_entropy = 0.808095
Epoch 169
Loss = 8.5258e-02, PNorm = 61.1109, GNorm = 7.0135, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.758124
Epoch 170
Validation binary_cross_entropy = 0.731575
Epoch 171
Validation binary_cross_entropy = 0.755262
Epoch 172
Validation binary_cross_entropy = 0.796789
Epoch 173
Validation binary_cross_entropy = 0.801351
Epoch 174
Loss = 2.9448e-02, PNorm = 61.2493, GNorm = 3.9360, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.878607
Epoch 175
Validation binary_cross_entropy = 0.891325
Epoch 176
Validation binary_cross_entropy = 0.926922
Epoch 177
Validation binary_cross_entropy = 0.952878
Epoch 178
Validation binary_cross_entropy = 0.981793
Epoch 179
Loss = 6.7016e-02, PNorm = 61.3586, GNorm = 3.1455, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.910424
Epoch 180
Validation binary_cross_entropy = 0.875331
Epoch 181
Validation binary_cross_entropy = 0.885113
Epoch 182
Validation binary_cross_entropy = 0.925885
Epoch 183
Validation binary_cross_entropy = 0.980355
Epoch 184
Loss = 1.3235e-01, PNorm = 61.4701, GNorm = 10.8698, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.057108
Epoch 185
Validation binary_cross_entropy = 1.106393
Epoch 186
Validation binary_cross_entropy = 1.119010
Epoch 187
Validation binary_cross_entropy = 1.246107
Epoch 188
Validation binary_cross_entropy = 1.111269
Epoch 189
Loss = 1.0280e-02, PNorm = 61.5983, GNorm = 1.4977, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.065553
Epoch 190
Validation binary_cross_entropy = 1.054205
Epoch 191
Validation binary_cross_entropy = 1.101583
Epoch 192
Validation binary_cross_entropy = 1.038852
Epoch 193
Validation binary_cross_entropy = 1.038169
Epoch 194
Loss = 1.5374e-02, PNorm = 61.6809, GNorm = 0.8854, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.043631
Epoch 195
Validation binary_cross_entropy = 1.035091
Epoch 196
Validation binary_cross_entropy = 1.027218
Epoch 197
Validation binary_cross_entropy = 1.030244
Epoch 198
Validation binary_cross_entropy = 1.034168
Epoch 199
Loss = 2.7706e-02, PNorm = 61.7531, GNorm = 3.4335, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.050314
Epoch 200
Validation binary_cross_entropy = 1.128623
Epoch 201
Validation binary_cross_entropy = 1.270626
Epoch 202
Validation binary_cross_entropy = 1.111794
Epoch 203
Validation binary_cross_entropy = 1.116988
Epoch 204
Loss = 7.9063e-02, PNorm = 61.8330, GNorm = 3.7023, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.106283
Epoch 205
Validation binary_cross_entropy = 1.074921
Epoch 206
Validation binary_cross_entropy = 1.182968
Epoch 207
Validation binary_cross_entropy = 1.145274
Epoch 208
Validation binary_cross_entropy = 1.030345
Epoch 209
Loss = 5.9463e-03, PNorm = 62.0030, GNorm = 0.6085, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.020955
Epoch 210
Validation binary_cross_entropy = 1.070608
Epoch 211
Validation binary_cross_entropy = 1.082119
Epoch 212
Validation binary_cross_entropy = 1.178543
Epoch 213
Validation binary_cross_entropy = 1.211292
Epoch 214
Loss = 3.0972e-02, PNorm = 62.1431, GNorm = 0.9936, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.199138
Epoch 215
Validation binary_cross_entropy = 1.283790
Epoch 216
Validation binary_cross_entropy = 1.133498
Epoch 217
Validation binary_cross_entropy = 1.102913
Epoch 218
Validation binary_cross_entropy = 1.110696
Epoch 219
Loss = 1.6237e-02, PNorm = 62.2363, GNorm = 0.7187, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.105516
Epoch 220
Validation binary_cross_entropy = 1.073801
Epoch 221
Validation binary_cross_entropy = 1.116466
Epoch 222
Validation binary_cross_entropy = 1.184107
Epoch 223
Validation binary_cross_entropy = 1.240553
Epoch 224
Loss = 3.9881e-02, PNorm = 62.3219, GNorm = 2.2127, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.296217
Epoch 225
Validation binary_cross_entropy = 1.330688
Epoch 226
Validation binary_cross_entropy = 1.331984
Epoch 227
Validation binary_cross_entropy = 1.350170
Epoch 228
Validation binary_cross_entropy = 1.329989
Epoch 229
Loss = 2.1542e-02, PNorm = 62.3998, GNorm = 0.3687, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.255521
Epoch 230
Validation binary_cross_entropy = 1.223390
Epoch 231
Validation binary_cross_entropy = 1.238040
Epoch 232
Validation binary_cross_entropy = 1.258147
Epoch 233
Validation binary_cross_entropy = 1.249116
Epoch 234
Loss = 5.3850e-02, PNorm = 62.4720, GNorm = 0.6590, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.198005
Epoch 235
Validation binary_cross_entropy = 1.182899
Epoch 236
Validation binary_cross_entropy = 1.191265
Epoch 237
Validation binary_cross_entropy = 1.221248
Epoch 238
Validation binary_cross_entropy = 1.244724
Epoch 239
Loss = 2.7212e-02, PNorm = 62.5341, GNorm = 1.2496, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.274397
Epoch 240
Validation binary_cross_entropy = 1.276107
Epoch 241
Validation binary_cross_entropy = 1.260467
Epoch 242
Validation binary_cross_entropy = 1.239814
Epoch 243
Validation binary_cross_entropy = 1.236199
Epoch 244
Loss = 3.3083e-03, PNorm = 62.5741, GNorm = 0.5259, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.239875
Epoch 245
Validation binary_cross_entropy = 1.277241
Epoch 246
Validation binary_cross_entropy = 1.321985
Epoch 247
Validation binary_cross_entropy = 1.340496
Epoch 248
Validation binary_cross_entropy = 1.340151
Epoch 249
Loss = 1.1214e-02, PNorm = 62.6034, GNorm = 1.5168, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.330449
Epoch 250
Validation binary_cross_entropy = 1.314405
Epoch 251
Validation binary_cross_entropy = 1.340984
Epoch 252
Validation binary_cross_entropy = 1.391470
Epoch 253
Validation binary_cross_entropy = 1.461560
Epoch 254
Loss = 1.3675e-02, PNorm = 62.6538, GNorm = 2.7927, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.470318
Epoch 255
Validation binary_cross_entropy = 1.471103
Epoch 256
Validation binary_cross_entropy = 1.490225
Epoch 257
Validation binary_cross_entropy = 1.476914
Epoch 258
Validation binary_cross_entropy = 1.564831
Epoch 259
Loss = 6.8952e-03, PNorm = 62.7297, GNorm = 0.4871, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.656108
Epoch 260
Validation binary_cross_entropy = 1.569622
Epoch 261
Validation binary_cross_entropy = 1.514993
Epoch 262
Validation binary_cross_entropy = 1.498008
Epoch 263
Validation binary_cross_entropy = 1.521519
Epoch 264
Loss = 1.4306e-02, PNorm = 62.7964, GNorm = 0.4648, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.599804
Epoch 265
Validation binary_cross_entropy = 1.649550
Epoch 266
Validation binary_cross_entropy = 1.527384
Epoch 267
Validation binary_cross_entropy = 1.491708
Epoch 268
Validation binary_cross_entropy = 1.494254
Epoch 269
Loss = 3.8347e-02, PNorm = 62.8589, GNorm = 4.7284, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.467422
Epoch 270
Validation binary_cross_entropy = 1.474817
Epoch 271
Validation binary_cross_entropy = 1.477379
Epoch 272
Validation binary_cross_entropy = 1.446881
Epoch 273
Validation binary_cross_entropy = 1.409752
Epoch 274
Loss = 1.9087e-02, PNorm = 62.8987, GNorm = 1.6777, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.421718
Epoch 275
Validation binary_cross_entropy = 1.448243
Epoch 276
Validation binary_cross_entropy = 1.473280
Epoch 277
Validation binary_cross_entropy = 1.484973
Epoch 278
Validation binary_cross_entropy = 1.493530
Epoch 279
Loss = 2.2654e-03, PNorm = 62.9261, GNorm = 0.2165, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.500289
Epoch 280
Validation binary_cross_entropy = 1.530984
Epoch 281
Validation binary_cross_entropy = 1.524749
Epoch 282
Validation binary_cross_entropy = 1.509405
Epoch 283
Validation binary_cross_entropy = 1.490525
Epoch 284
Loss = 1.0631e-01, PNorm = 62.9629, GNorm = 10.4378, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.467721
Epoch 285
Validation binary_cross_entropy = 1.458272
Epoch 286
Validation binary_cross_entropy = 1.449172
Epoch 287
Validation binary_cross_entropy = 1.442997
Epoch 288
Validation binary_cross_entropy = 1.446414
Epoch 289
Loss = 5.0084e-03, PNorm = 63.0053, GNorm = 0.3348, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.443456
Epoch 290
Validation binary_cross_entropy = 1.438203
Epoch 291
Validation binary_cross_entropy = 1.427670
Epoch 292
Validation binary_cross_entropy = 1.428235
Epoch 293
Validation binary_cross_entropy = 1.430717
Epoch 294
Loss = 4.7131e-02, PNorm = 63.0462, GNorm = 7.3771, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.426340
Epoch 295
Validation binary_cross_entropy = 1.524104
Epoch 296
Validation binary_cross_entropy = 1.594627
Epoch 297
Validation binary_cross_entropy = 1.559150
Epoch 298
Validation binary_cross_entropy = 1.495232
Epoch 299
Loss = 3.2685e-03, PNorm = 63.1111, GNorm = 0.2300, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.478901
Model 0 best validation binary_cross_entropy = 0.335619 on epoch 30
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.203085
Ensemble test binary_cross_entropy = 0.203085
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.515195
Epoch 1
Validation binary_cross_entropy = 0.507515
Epoch 2
Validation binary_cross_entropy = 0.893476
Epoch 3
Validation binary_cross_entropy = 1.045011
Epoch 4
Loss = 1.6125e+00, PNorm = 54.1611, GNorm = 13.4334, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.672615
Epoch 5
Validation binary_cross_entropy = 0.448408
Epoch 6
Validation binary_cross_entropy = 0.438646
Epoch 7
Validation binary_cross_entropy = 0.644250
Epoch 8
Validation binary_cross_entropy = 0.720906
Epoch 9
Loss = 1.3509e+00, PNorm = 54.3276, GNorm = 11.1314, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.520650
Epoch 10
Validation binary_cross_entropy = 0.458351
Epoch 11
Validation binary_cross_entropy = 0.496642
Epoch 12
Validation binary_cross_entropy = 0.675996
Epoch 13
Validation binary_cross_entropy = 0.878015
Epoch 14
Loss = 1.2878e+00, PNorm = 54.4889, GNorm = 14.4816, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.732734
Epoch 15
Validation binary_cross_entropy = 0.503865
Epoch 16
Validation binary_cross_entropy = 0.415866
Epoch 17
Validation binary_cross_entropy = 0.407227
Epoch 18
Validation binary_cross_entropy = 0.456700
Epoch 19
Loss = 1.2363e+00, PNorm = 54.6110, GNorm = 18.2127, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.467533
Epoch 20
Validation binary_cross_entropy = 0.526646
Epoch 21
Validation binary_cross_entropy = 0.591801
Epoch 22
Validation binary_cross_entropy = 0.498795
Epoch 23
Validation binary_cross_entropy = 0.386112
Epoch 24
Loss = 1.0892e+00, PNorm = 54.7393, GNorm = 15.5487, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.438211
Epoch 25
Validation binary_cross_entropy = 0.616633
Epoch 26
Validation binary_cross_entropy = 0.425021
Epoch 27
Validation binary_cross_entropy = 0.341435
Epoch 28
Validation binary_cross_entropy = 0.528043
Epoch 29
Loss = 8.2128e-01, PNorm = 54.8848, GNorm = 25.7705, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.478008
Epoch 30
Validation binary_cross_entropy = 0.389695
Epoch 31
Validation binary_cross_entropy = 0.432765
Epoch 32
Validation binary_cross_entropy = 0.485521
Epoch 33
Validation binary_cross_entropy = 0.491899
Epoch 34
Loss = 1.0245e+00, PNorm = 55.0141, GNorm = 13.6019, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.407154
Epoch 35
Validation binary_cross_entropy = 0.425597
Epoch 36
Validation binary_cross_entropy = 0.408411
Epoch 37
Validation binary_cross_entropy = 0.369838
Epoch 38
Validation binary_cross_entropy = 0.364887
Epoch 39
Loss = 9.8078e-01, PNorm = 55.1800, GNorm = 21.9429, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.408219
Epoch 40
Validation binary_cross_entropy = 0.389238
Epoch 41
Validation binary_cross_entropy = 0.415586
Epoch 42
Validation binary_cross_entropy = 0.376904
Epoch 43
Validation binary_cross_entropy = 0.374694
Epoch 44
Loss = 6.1931e-01, PNorm = 55.3128, GNorm = 13.1211, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.549469
Epoch 45
Validation binary_cross_entropy = 0.369173
Epoch 46
Validation binary_cross_entropy = 0.367705
Epoch 47
Validation binary_cross_entropy = 0.608052
Epoch 48
Validation binary_cross_entropy = 0.354476
Epoch 49
Loss = 6.6267e-01, PNorm = 55.4967, GNorm = 23.5765, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.370214
Epoch 50
Validation binary_cross_entropy = 0.396667
Epoch 51
Validation binary_cross_entropy = 0.368428
Epoch 52
Validation binary_cross_entropy = 0.457993
Epoch 53
Validation binary_cross_entropy = 0.401768
Epoch 54
Loss = 6.6504e-01, PNorm = 55.7218, GNorm = 21.2150, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.395287
Epoch 55
Validation binary_cross_entropy = 0.356467
Epoch 56
Validation binary_cross_entropy = 0.475378
Epoch 57
Validation binary_cross_entropy = 0.390420
Epoch 58
Validation binary_cross_entropy = 0.356464
Epoch 59
Loss = 6.5973e-01, PNorm = 55.9561, GNorm = 9.1122, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.589000
Epoch 60
Validation binary_cross_entropy = 0.481501
Epoch 61
Validation binary_cross_entropy = 0.385077
Epoch 62
Validation binary_cross_entropy = 0.409439
Epoch 63
Validation binary_cross_entropy = 0.523247
Epoch 64
Loss = 7.0206e-01, PNorm = 56.2235, GNorm = 14.2047, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.460811
Epoch 65
Validation binary_cross_entropy = 0.403452
Epoch 66
Validation binary_cross_entropy = 0.637393
Epoch 67
Validation binary_cross_entropy = 0.387468
Epoch 68
Validation binary_cross_entropy = 0.383886
Epoch 69
Loss = 5.8836e-01, PNorm = 56.4971, GNorm = 10.4973, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.460534
Epoch 70
Validation binary_cross_entropy = 0.411931
Epoch 71
Validation binary_cross_entropy = 0.405598
Epoch 72
Validation binary_cross_entropy = 0.410149
Epoch 73
Validation binary_cross_entropy = 0.534998
Epoch 74
Loss = 3.4631e-01, PNorm = 56.7926, GNorm = 8.3727, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.413279
Epoch 75
Validation binary_cross_entropy = 0.460170
Epoch 76
Validation binary_cross_entropy = 0.800284
Epoch 77
Validation binary_cross_entropy = 0.455472
Epoch 78
Validation binary_cross_entropy = 0.459858
Epoch 79
Loss = 2.8367e-01, PNorm = 57.0720, GNorm = 10.6707, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.619988
Epoch 80
Validation binary_cross_entropy = 0.600220
Epoch 81
Validation binary_cross_entropy = 0.473366
Epoch 82
Validation binary_cross_entropy = 0.467033
Epoch 83
Validation binary_cross_entropy = 0.529023
Epoch 84
Loss = 4.9259e-01, PNorm = 57.3646, GNorm = 10.6114, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.656190
Epoch 85
Validation binary_cross_entropy = 0.449769
Epoch 86
Validation binary_cross_entropy = 0.440787
Epoch 87
Validation binary_cross_entropy = 0.441053
Epoch 88
Validation binary_cross_entropy = 0.427634
Epoch 89
Loss = 4.9552e-01, PNorm = 57.6507, GNorm = 8.4033, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.495182
Epoch 90
Validation binary_cross_entropy = 0.546107
Epoch 91
Validation binary_cross_entropy = 0.557652
Epoch 92
Validation binary_cross_entropy = 0.522430
Epoch 93
Validation binary_cross_entropy = 0.538826
Epoch 94
Loss = 3.4520e-01, PNorm = 57.9461, GNorm = 13.4731, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.626919
Epoch 95
Validation binary_cross_entropy = 0.553370
Epoch 96
Validation binary_cross_entropy = 0.551467
Epoch 97
Validation binary_cross_entropy = 0.554312
Epoch 98
Validation binary_cross_entropy = 0.568272
Epoch 99
Loss = 1.7900e-01, PNorm = 58.2411, GNorm = 4.4539, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.551141
Epoch 100
Validation binary_cross_entropy = 0.525239
Epoch 101
Validation binary_cross_entropy = 0.546063
Epoch 102
Validation binary_cross_entropy = 0.603274
Epoch 103
Validation binary_cross_entropy = 0.588870
Epoch 104
Loss = 1.3366e-01, PNorm = 58.4989, GNorm = 10.0842, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.750509
Epoch 105
Validation binary_cross_entropy = 0.623726
Epoch 106
Validation binary_cross_entropy = 0.657879
Epoch 107
Validation binary_cross_entropy = 0.702135
Epoch 108
Validation binary_cross_entropy = 0.711176
Epoch 109
Loss = 2.8323e-01, PNorm = 58.7365, GNorm = 12.7739, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.663467
Epoch 110
Validation binary_cross_entropy = 0.629220
Epoch 111
Validation binary_cross_entropy = 0.660489
Epoch 112
Validation binary_cross_entropy = 0.545248
Epoch 113
Validation binary_cross_entropy = 0.483200
Epoch 114
Loss = 2.9093e-01, PNorm = 58.9923, GNorm = 17.5813, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.507360
Epoch 115
Validation binary_cross_entropy = 0.536035
Epoch 116
Validation binary_cross_entropy = 0.703288
Epoch 117
Validation binary_cross_entropy = 0.881537
Epoch 118
Validation binary_cross_entropy = 0.655060
Epoch 119
Loss = 1.7388e-01, PNorm = 59.2220, GNorm = 5.0553, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.627853
Epoch 120
Validation binary_cross_entropy = 0.590036
Epoch 121
Validation binary_cross_entropy = 0.585792
Epoch 122
Validation binary_cross_entropy = 0.595866
Epoch 123
Validation binary_cross_entropy = 0.632569
Epoch 124
Loss = 3.3813e-01, PNorm = 59.4207, GNorm = 16.3314, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.629794
Epoch 125
Validation binary_cross_entropy = 0.645732
Epoch 126
Validation binary_cross_entropy = 0.661504
Epoch 127
Validation binary_cross_entropy = 0.645629
Epoch 128
Validation binary_cross_entropy = 0.680845
Epoch 129
Loss = 2.1918e-01, PNorm = 59.5999, GNorm = 18.1061, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.648889
Epoch 130
Validation binary_cross_entropy = 0.793402
Epoch 131
Validation binary_cross_entropy = 0.699516
Epoch 132
Validation binary_cross_entropy = 0.653009
Epoch 133
Validation binary_cross_entropy = 0.677290
Epoch 134
Loss = 1.3388e-01, PNorm = 59.7838, GNorm = 15.1012, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.596011
Epoch 135
Validation binary_cross_entropy = 0.581069
Epoch 136
Validation binary_cross_entropy = 0.677553
Epoch 137
Validation binary_cross_entropy = 0.602009
Epoch 138
Validation binary_cross_entropy = 0.672667
Epoch 139
Loss = 2.1388e-01, PNorm = 59.9626, GNorm = 7.2890, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.749326
Epoch 140
Validation binary_cross_entropy = 0.752461
Epoch 141
Validation binary_cross_entropy = 0.939959
Epoch 142
Validation binary_cross_entropy = 0.835384
Epoch 143
Validation binary_cross_entropy = 0.765870
Epoch 144
Loss = 1.5765e-01, PNorm = 60.1404, GNorm = 14.3150, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.757586
Epoch 145
Validation binary_cross_entropy = 0.659586
Epoch 146
Validation binary_cross_entropy = 0.646235
Epoch 147
Validation binary_cross_entropy = 0.636812
Epoch 148
Validation binary_cross_entropy = 0.632478
Epoch 149
Loss = 7.5619e-02, PNorm = 60.3295, GNorm = 8.0024, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.660124
Epoch 150
Validation binary_cross_entropy = 0.676405
Epoch 151
Validation binary_cross_entropy = 0.712255
Epoch 152
Validation binary_cross_entropy = 0.773692
Epoch 153
Validation binary_cross_entropy = 0.806913
Epoch 154
Loss = 7.7126e-02, PNorm = 60.4959, GNorm = 2.7132, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.815780
Epoch 155
Validation binary_cross_entropy = 0.841251
Epoch 156
Validation binary_cross_entropy = 0.862540
Epoch 157
Validation binary_cross_entropy = 0.856872
Epoch 158
Validation binary_cross_entropy = 0.879534
Epoch 159
Loss = 8.2454e-02, PNorm = 60.6541, GNorm = 1.5066, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.867252
Epoch 160
Validation binary_cross_entropy = 0.881399
Epoch 161
Validation binary_cross_entropy = 0.877981
Epoch 162
Validation binary_cross_entropy = 0.881313
Epoch 163
Validation binary_cross_entropy = 0.888816
Epoch 164
Loss = 5.2188e-02, PNorm = 60.7787, GNorm = 2.4129, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.901342
Epoch 165
Validation binary_cross_entropy = 0.899246
Epoch 166
Validation binary_cross_entropy = 0.886967
Epoch 167
Validation binary_cross_entropy = 0.885704
Epoch 168
Validation binary_cross_entropy = 0.867450
Epoch 169
Loss = 6.1306e-02, PNorm = 60.8905, GNorm = 1.5059, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.856845
Epoch 170
Validation binary_cross_entropy = 0.843047
Epoch 171
Validation binary_cross_entropy = 0.833072
Epoch 172
Validation binary_cross_entropy = 0.858585
Epoch 173
Validation binary_cross_entropy = 0.822652
Epoch 174
Loss = 5.4790e-02, PNorm = 60.9785, GNorm = 4.0431, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.837610
Epoch 175
Validation binary_cross_entropy = 0.867366
Epoch 176
Validation binary_cross_entropy = 0.874347
Epoch 177
Validation binary_cross_entropy = 0.893383
Epoch 178
Validation binary_cross_entropy = 0.898529
Epoch 179
Loss = 6.2530e-02, PNorm = 61.0945, GNorm = 2.8962, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.912254
Epoch 180
Validation binary_cross_entropy = 0.947265
Epoch 181
Validation binary_cross_entropy = 0.979859
Epoch 182
Validation binary_cross_entropy = 1.005767
Epoch 183
Validation binary_cross_entropy = 1.016119
Epoch 184
Loss = 1.0733e-01, PNorm = 61.2137, GNorm = 3.9898, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.052823
Epoch 185
Validation binary_cross_entropy = 1.032028
Epoch 186
Validation binary_cross_entropy = 0.976967
Epoch 187
Validation binary_cross_entropy = 0.950742
Epoch 188
Validation binary_cross_entropy = 0.933361
Epoch 189
Loss = 4.6319e-02, PNorm = 61.3278, GNorm = 5.2364, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.924879
Epoch 190
Validation binary_cross_entropy = 0.928545
Epoch 191
Validation binary_cross_entropy = 1.003523
Epoch 192
Validation binary_cross_entropy = 1.121665
Epoch 193
Validation binary_cross_entropy = 1.011820
Epoch 194
Loss = 6.5688e-02, PNorm = 61.4264, GNorm = 10.7245, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.045000
Epoch 195
Validation binary_cross_entropy = 1.007413
Epoch 196
Validation binary_cross_entropy = 1.060366
Epoch 197
Validation binary_cross_entropy = 1.067799
Epoch 198
Validation binary_cross_entropy = 0.988319
Epoch 199
Loss = 1.0710e-02, PNorm = 61.5549, GNorm = 0.7661, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.974534
Epoch 200
Validation binary_cross_entropy = 0.962165
Epoch 201
Validation binary_cross_entropy = 0.900932
Epoch 202
Validation binary_cross_entropy = 0.873432
Epoch 203
Validation binary_cross_entropy = 0.853965
Epoch 204
Loss = 2.0062e-02, PNorm = 61.6602, GNorm = 0.6517, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.847827
Epoch 205
Validation binary_cross_entropy = 0.862425
Epoch 206
Validation binary_cross_entropy = 0.877224
Epoch 207
Validation binary_cross_entropy = 0.898192
Epoch 208
Validation binary_cross_entropy = 0.933230
Epoch 209
Loss = 1.5034e-02, PNorm = 61.7404, GNorm = 1.3528, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.953528
Epoch 210
Validation binary_cross_entropy = 0.928108
Epoch 211
Validation binary_cross_entropy = 0.972277
Epoch 212
Validation binary_cross_entropy = 0.921738
Epoch 213
Validation binary_cross_entropy = 0.928431
Epoch 214
Loss = 1.0260e-01, PNorm = 61.8545, GNorm = 12.7568, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.047570
Epoch 215
Validation binary_cross_entropy = 0.964425
Epoch 216
Validation binary_cross_entropy = 0.836419
Epoch 217
Validation binary_cross_entropy = 0.836056
Epoch 218
Validation binary_cross_entropy = 0.868638
Epoch 219
Loss = 1.6426e-02, PNorm = 61.9886, GNorm = 1.7096, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.890770
Epoch 220
Validation binary_cross_entropy = 0.902253
Epoch 221
Validation binary_cross_entropy = 0.909940
Epoch 222
Validation binary_cross_entropy = 0.944345
Epoch 223
Validation binary_cross_entropy = 0.996902
Epoch 224
Loss = 3.1457e-02, PNorm = 62.0856, GNorm = 2.3735, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.052689
Epoch 225
Validation binary_cross_entropy = 1.101567
Epoch 226
Validation binary_cross_entropy = 1.130609
Epoch 227
Validation binary_cross_entropy = 1.151525
Epoch 228
Validation binary_cross_entropy = 1.145696
Epoch 229
Loss = 4.8160e-03, PNorm = 62.1500, GNorm = 0.5451, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.156121
Epoch 230
Validation binary_cross_entropy = 1.164084
Epoch 231
Validation binary_cross_entropy = 1.150525
Epoch 232
Validation binary_cross_entropy = 1.156846
Epoch 233
Validation binary_cross_entropy = 1.178871
Epoch 234
Loss = 6.8367e-02, PNorm = 62.2115, GNorm = 11.1632, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.163461
Epoch 235
Validation binary_cross_entropy = 1.162433
Epoch 236
Validation binary_cross_entropy = 1.191315
Epoch 237
Validation binary_cross_entropy = 1.234632
Epoch 238
Validation binary_cross_entropy = 1.269359
Epoch 239
Loss = 2.7936e-02, PNorm = 62.3066, GNorm = 0.6937, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.304108
Epoch 240
Validation binary_cross_entropy = 1.380228
Epoch 241
Validation binary_cross_entropy = 1.348814
Epoch 242
Validation binary_cross_entropy = 1.257575
Epoch 243
Validation binary_cross_entropy = 1.260931
Epoch 244
Loss = 1.2591e-01, PNorm = 62.3904, GNorm = 12.9722, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.179181
Epoch 245
Validation binary_cross_entropy = 1.188612
Epoch 246
Validation binary_cross_entropy = 1.190239
Epoch 247
Validation binary_cross_entropy = 1.062577
Epoch 248
Validation binary_cross_entropy = 1.067868
Epoch 249
Loss = 4.5334e-02, PNorm = 62.5140, GNorm = 0.1851, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.108278
Epoch 250
Validation binary_cross_entropy = 1.168508
Epoch 251
Validation binary_cross_entropy = 1.205193
Epoch 252
Validation binary_cross_entropy = 1.246099
Epoch 253
Validation binary_cross_entropy = 1.238780
Epoch 254
Loss = 3.2521e-02, PNorm = 62.6225, GNorm = 4.0818, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.222740
Epoch 255
Validation binary_cross_entropy = 1.217168
Epoch 256
Validation binary_cross_entropy = 1.226041
Epoch 257
Validation binary_cross_entropy = 1.221198
Epoch 258
Validation binary_cross_entropy = 1.237771
Epoch 259
Loss = 3.4089e-02, PNorm = 62.7026, GNorm = 4.7863, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.160115
Epoch 260
Validation binary_cross_entropy = 1.077121
Epoch 261
Validation binary_cross_entropy = 1.071747
Epoch 262
Validation binary_cross_entropy = 1.125668
Epoch 263
Validation binary_cross_entropy = 1.208401
Epoch 264
Loss = 2.7866e-02, PNorm = 62.7890, GNorm = 8.5199, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.296691
Epoch 265
Validation binary_cross_entropy = 1.291338
Epoch 266
Validation binary_cross_entropy = 1.273896
Epoch 267
Validation binary_cross_entropy = 1.303409
Epoch 268
Validation binary_cross_entropy = 1.299225
Epoch 269
Loss = 2.0197e-02, PNorm = 62.9010, GNorm = 2.0091, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.313519
Epoch 270
Validation binary_cross_entropy = 1.265460
Epoch 271
Validation binary_cross_entropy = 1.257096
Epoch 272
Validation binary_cross_entropy = 1.279945
Epoch 273
Validation binary_cross_entropy = 1.293520
Epoch 274
Loss = 3.3325e-02, PNorm = 62.9792, GNorm = 3.3298, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.311699
Epoch 275
Validation binary_cross_entropy = 1.335457
Epoch 276
Validation binary_cross_entropy = 1.340477
Epoch 277
Validation binary_cross_entropy = 1.352832
Epoch 278
Validation binary_cross_entropy = 1.382067
Epoch 279
Loss = 3.2410e-02, PNorm = 63.0301, GNorm = 0.4049, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.383921
Epoch 280
Validation binary_cross_entropy = 1.408675
Epoch 281
Validation binary_cross_entropy = 1.459761
Epoch 282
Validation binary_cross_entropy = 1.524398
Epoch 283
Validation binary_cross_entropy = 1.464193
Epoch 284
Loss = 1.0661e-02, PNorm = 63.0759, GNorm = 0.4476, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.406289
Epoch 285
Validation binary_cross_entropy = 1.383125
Epoch 286
Validation binary_cross_entropy = 1.340190
Epoch 287
Validation binary_cross_entropy = 1.293812
Epoch 288
Validation binary_cross_entropy = 1.273179
Epoch 289
Loss = 6.0401e-02, PNorm = 63.1293, GNorm = 15.0221, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.282120
Epoch 290
Validation binary_cross_entropy = 1.306134
Epoch 291
Validation binary_cross_entropy = 1.364009
Epoch 292
Validation binary_cross_entropy = 1.425282
Epoch 293
Validation binary_cross_entropy = 1.492695
Epoch 294
Loss = 9.2904e-03, PNorm = 63.1993, GNorm = 0.4673, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.541056
Epoch 295
Validation binary_cross_entropy = 1.571688
Epoch 296
Validation binary_cross_entropy = 1.589578
Epoch 297
Validation binary_cross_entropy = 1.536403
Epoch 298
Validation binary_cross_entropy = 1.509402
Epoch 299
Loss = 5.2868e-03, PNorm = 63.2581, GNorm = 1.0882, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.495432
Model 0 best validation binary_cross_entropy = 0.341435 on epoch 27
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.255989
Ensemble test binary_cross_entropy = 0.255989
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.488820
Epoch 1
Validation binary_cross_entropy = 0.446731
Epoch 2
Validation binary_cross_entropy = 0.597887
Epoch 3
Validation binary_cross_entropy = 0.839252
Epoch 4
Loss = 1.5300e+00, PNorm = 54.1646, GNorm = 22.7001, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.711199
Epoch 5
Validation binary_cross_entropy = 0.476158
Epoch 6
Validation binary_cross_entropy = 0.446190
Epoch 7
Validation binary_cross_entropy = 0.544304
Epoch 8
Validation binary_cross_entropy = 0.847301
Epoch 9
Loss = 1.3855e+00, PNorm = 54.3628, GNorm = 19.2116, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.713156
Epoch 10
Validation binary_cross_entropy = 0.524253
Epoch 11
Validation binary_cross_entropy = 0.462915
Epoch 12
Validation binary_cross_entropy = 0.484729
Epoch 13
Validation binary_cross_entropy = 0.704573
Epoch 14
Loss = 1.3440e+00, PNorm = 54.5563, GNorm = 10.1096, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.777834
Epoch 15
Validation binary_cross_entropy = 0.604970
Epoch 16
Validation binary_cross_entropy = 0.473031
Epoch 17
Validation binary_cross_entropy = 0.372242
Epoch 18
Validation binary_cross_entropy = 0.382876
Epoch 19
Loss = 1.4329e+00, PNorm = 54.6991, GNorm = 33.0371, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.498679
Epoch 20
Validation binary_cross_entropy = 0.803603
Epoch 21
Validation binary_cross_entropy = 0.848264
Epoch 22
Validation binary_cross_entropy = 0.572771
Epoch 23
Validation binary_cross_entropy = 0.404353
Epoch 24
Loss = 8.7677e-01, PNorm = 54.8185, GNorm = 12.1620, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.420652
Epoch 25
Validation binary_cross_entropy = 0.540668
Epoch 26
Validation binary_cross_entropy = 0.802241
Epoch 27
Validation binary_cross_entropy = 0.567973
Epoch 28
Validation binary_cross_entropy = 0.376163
Epoch 29
Loss = 1.3233e+00, PNorm = 54.9284, GNorm = 21.7360, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.367307
Epoch 30
Validation binary_cross_entropy = 0.371900
Epoch 31
Validation binary_cross_entropy = 0.499433
Epoch 32
Validation binary_cross_entropy = 0.746659
Epoch 33
Validation binary_cross_entropy = 0.509933
Epoch 34
Loss = 9.0604e-01, PNorm = 55.0454, GNorm = 7.4670, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.378877
Epoch 35
Validation binary_cross_entropy = 0.349401
Epoch 36
Validation binary_cross_entropy = 0.385859
Epoch 37
Validation binary_cross_entropy = 0.447041
Epoch 38
Validation binary_cross_entropy = 0.485711
Epoch 39
Loss = 7.9914e-01, PNorm = 55.1757, GNorm = 29.3608, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.393556
Epoch 40
Validation binary_cross_entropy = 0.346249
Epoch 41
Validation binary_cross_entropy = 0.396665
Epoch 42
Validation binary_cross_entropy = 0.709027
Epoch 43
Validation binary_cross_entropy = 0.510139
Epoch 44
Loss = 6.0745e-01, PNorm = 55.3392, GNorm = 17.5232, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.358566
Epoch 45
Validation binary_cross_entropy = 0.355287
Epoch 46
Validation binary_cross_entropy = 0.431360
Epoch 47
Validation binary_cross_entropy = 0.560118
Epoch 48
Validation binary_cross_entropy = 0.515078
Epoch 49
Loss = 8.5893e-01, PNorm = 55.4985, GNorm = 12.8240, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.379736
Epoch 50
Validation binary_cross_entropy = 0.371262
Epoch 51
Validation binary_cross_entropy = 0.368559
Epoch 52
Validation binary_cross_entropy = 0.410563
Epoch 53
Validation binary_cross_entropy = 0.681112
Epoch 54
Loss = 8.8141e-01, PNorm = 55.7009, GNorm = 12.4949, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.471576
Epoch 55
Validation binary_cross_entropy = 0.361328
Epoch 56
Validation binary_cross_entropy = 0.359082
Epoch 57
Validation binary_cross_entropy = 0.362956
Epoch 58
Validation binary_cross_entropy = 0.407641
Epoch 59
Loss = 6.6008e-01, PNorm = 55.8971, GNorm = 14.1704, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.407092
Epoch 60
Validation binary_cross_entropy = 0.404173
Epoch 61
Validation binary_cross_entropy = 0.385710
Epoch 62
Validation binary_cross_entropy = 0.431409
Epoch 63
Validation binary_cross_entropy = 0.483541
Epoch 64
Loss = 6.9631e-01, PNorm = 56.0977, GNorm = 8.2648, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.466785
Epoch 65
Validation binary_cross_entropy = 0.457759
Epoch 66
Validation binary_cross_entropy = 0.475941
Epoch 67
Validation binary_cross_entropy = 0.551031
Epoch 68
Validation binary_cross_entropy = 0.389925
Epoch 69
Loss = 4.6148e-01, PNorm = 56.3247, GNorm = 5.1846, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.403933
Epoch 70
Validation binary_cross_entropy = 0.433442
Epoch 71
Validation binary_cross_entropy = 0.461203
Epoch 72
Validation binary_cross_entropy = 0.377020
Epoch 73
Validation binary_cross_entropy = 0.381789
Epoch 74
Loss = 5.9344e-01, PNorm = 56.5775, GNorm = 9.3954, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.517588
Epoch 75
Validation binary_cross_entropy = 0.493204
Epoch 76
Validation binary_cross_entropy = 0.407676
Epoch 77
Validation binary_cross_entropy = 0.421190
Epoch 78
Validation binary_cross_entropy = 0.595370
Epoch 79
Loss = 3.3358e-01, PNorm = 56.8940, GNorm = 6.0098, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601815
Epoch 80
Validation binary_cross_entropy = 0.405481
Epoch 81
Validation binary_cross_entropy = 0.385675
Epoch 82
Validation binary_cross_entropy = 0.432437
Epoch 83
Validation binary_cross_entropy = 0.614003
Epoch 84
Loss = 5.3013e-01, PNorm = 57.2132, GNorm = 31.0479, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.460410
Epoch 85
Validation binary_cross_entropy = 0.413532
Epoch 86
Validation binary_cross_entropy = 0.441005
Epoch 87
Validation binary_cross_entropy = 0.478625
Epoch 88
Validation binary_cross_entropy = 0.552122
Epoch 89
Loss = 3.8715e-01, PNorm = 57.5469, GNorm = 19.5828, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.515037
Epoch 90
Validation binary_cross_entropy = 0.503133
Epoch 91
Validation binary_cross_entropy = 0.530359
Epoch 92
Validation binary_cross_entropy = 0.622477
Epoch 93
Validation binary_cross_entropy = 0.762425
Epoch 94
Loss = 1.5426e-01, PNorm = 57.8495, GNorm = 6.6714, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.699906
Epoch 95
Validation binary_cross_entropy = 0.517372
Epoch 96
Validation binary_cross_entropy = 0.473033
Epoch 97
Validation binary_cross_entropy = 0.547613
Epoch 98
Validation binary_cross_entropy = 0.488397
Epoch 99
Loss = 1.4896e-01, PNorm = 58.1536, GNorm = 6.8519, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.563805
Epoch 100
Validation binary_cross_entropy = 0.587891
Epoch 101
Validation binary_cross_entropy = 0.575867
Epoch 102
Validation binary_cross_entropy = 0.548300
Epoch 103
Validation binary_cross_entropy = 0.566538
Epoch 104
Loss = 2.9677e-01, PNorm = 58.4705, GNorm = 6.0554, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.570485
Epoch 105
Validation binary_cross_entropy = 0.577689
Epoch 106
Validation binary_cross_entropy = 0.563273
Epoch 107
Validation binary_cross_entropy = 0.613875
Epoch 108
Validation binary_cross_entropy = 0.679221
Epoch 109
Loss = 3.7982e-01, PNorm = 58.8040, GNorm = 14.8497, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.588182
Epoch 110
Validation binary_cross_entropy = 0.555627
Epoch 111
Validation binary_cross_entropy = 0.519507
Epoch 112
Validation binary_cross_entropy = 0.541542
Epoch 113
Validation binary_cross_entropy = 0.712928
Epoch 114
Loss = 3.5088e-01, PNorm = 59.1165, GNorm = 16.8472, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.801685
Epoch 115
Validation binary_cross_entropy = 0.606800
Epoch 116
Validation binary_cross_entropy = 0.654098
Epoch 117
Validation binary_cross_entropy = 0.659886
Epoch 118
Validation binary_cross_entropy = 0.812381
Epoch 119
Loss = 3.1537e-01, PNorm = 59.4376, GNorm = 6.2863, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.633861
Epoch 120
Validation binary_cross_entropy = 0.587238
Epoch 121
Validation binary_cross_entropy = 0.540532
Epoch 122
Validation binary_cross_entropy = 0.556803
Epoch 123
Validation binary_cross_entropy = 0.546383
Epoch 124
Loss = 3.7843e-01, PNorm = 59.7112, GNorm = 6.4779, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.597606
Epoch 125
Validation binary_cross_entropy = 0.613188
Epoch 126
Validation binary_cross_entropy = 0.639760
Epoch 127
Validation binary_cross_entropy = 0.683425
Epoch 128
Validation binary_cross_entropy = 0.700978
Epoch 129
Loss = 7.4086e-02, PNorm = 59.9819, GNorm = 6.1054, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.683883
Epoch 130
Validation binary_cross_entropy = 0.644436
Epoch 131
Validation binary_cross_entropy = 0.691718
Epoch 132
Validation binary_cross_entropy = 0.771897
Epoch 133
Validation binary_cross_entropy = 0.750505
Epoch 134
Loss = 2.2941e-01, PNorm = 60.2208, GNorm = 6.2025, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.765522
Epoch 135
Validation binary_cross_entropy = 0.815363
Epoch 136
Validation binary_cross_entropy = 0.668470
Epoch 137
Validation binary_cross_entropy = 0.630844
Epoch 138
Validation binary_cross_entropy = 0.652611
Epoch 139
Loss = 3.3609e-01, PNorm = 60.4568, GNorm = 4.2734, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.594289
Epoch 140
Validation binary_cross_entropy = 0.592683
Epoch 141
Validation binary_cross_entropy = 0.628840
Epoch 142
Validation binary_cross_entropy = 0.671688
Epoch 143
Validation binary_cross_entropy = 0.734605
Epoch 144
Loss = 1.2644e-01, PNorm = 60.6828, GNorm = 4.6454, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.759996
Epoch 145
Validation binary_cross_entropy = 0.836235
Epoch 146
Validation binary_cross_entropy = 0.889988
Epoch 147
Validation binary_cross_entropy = 0.904309
Epoch 148
Validation binary_cross_entropy = 0.913048
Epoch 149
Loss = 1.6929e-01, PNorm = 60.9003, GNorm = 4.9223, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.879171
Epoch 150
Validation binary_cross_entropy = 0.851163
Epoch 151
Validation binary_cross_entropy = 0.874746
Epoch 152
Validation binary_cross_entropy = 0.800381
Epoch 153
Validation binary_cross_entropy = 0.815383
Epoch 154
Loss = 1.9828e-01, PNorm = 61.0926, GNorm = 8.5813, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.774555
Epoch 155
Validation binary_cross_entropy = 0.746451
Epoch 156
Validation binary_cross_entropy = 0.749229
Epoch 157
Validation binary_cross_entropy = 0.757393
Epoch 158
Validation binary_cross_entropy = 0.794585
Epoch 159
Loss = 2.0155e-01, PNorm = 61.2943, GNorm = 9.8676, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.783109
Epoch 160
Validation binary_cross_entropy = 0.757413
Epoch 161
Validation binary_cross_entropy = 0.776435
Epoch 162
Validation binary_cross_entropy = 0.843437
Epoch 163
Validation binary_cross_entropy = 0.826413
Epoch 164
Loss = 8.2808e-02, PNorm = 61.4657, GNorm = 5.1449, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.841392
Epoch 165
Validation binary_cross_entropy = 0.864729
Epoch 166
Validation binary_cross_entropy = 0.865726
Epoch 167
Validation binary_cross_entropy = 0.847593
Epoch 168
Validation binary_cross_entropy = 0.823487
Epoch 169
Loss = 1.2396e-01, PNorm = 61.6357, GNorm = 2.9563, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.809731
Epoch 170
Validation binary_cross_entropy = 0.819805
Epoch 171
Validation binary_cross_entropy = 0.872559
Epoch 172
Validation binary_cross_entropy = 0.816244
Epoch 173
Validation binary_cross_entropy = 0.818345
Epoch 174
Loss = 2.2495e-01, PNorm = 61.8038, GNorm = 10.8255, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.821333
Epoch 175
Validation binary_cross_entropy = 0.880927
Epoch 176
Validation binary_cross_entropy = 0.936966
Epoch 177
Validation binary_cross_entropy = 0.829486
Epoch 178
Validation binary_cross_entropy = 0.807240
Epoch 179
Loss = 9.5119e-02, PNorm = 61.9360, GNorm = 4.0016, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.850625
Epoch 180
Validation binary_cross_entropy = 0.889337
Epoch 181
Validation binary_cross_entropy = 0.928719
Epoch 182
Validation binary_cross_entropy = 0.953784
Epoch 183
Validation binary_cross_entropy = 0.977991
Epoch 184
Loss = 8.3763e-02, PNorm = 62.0629, GNorm = 9.7583, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.967690
Epoch 185
Validation binary_cross_entropy = 0.942411
Epoch 186
Validation binary_cross_entropy = 0.945725
Epoch 187
Validation binary_cross_entropy = 0.952903
Epoch 188
Validation binary_cross_entropy = 0.981826
Epoch 189
Loss = 4.8869e-02, PNorm = 62.2142, GNorm = 1.8516, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.023197
Epoch 190
Validation binary_cross_entropy = 1.036327
Epoch 191
Validation binary_cross_entropy = 1.069354
Epoch 192
Validation binary_cross_entropy = 1.111696
Epoch 193
Validation binary_cross_entropy = 1.136502
Epoch 194
Loss = 5.1195e-02, PNorm = 62.3379, GNorm = 1.9426, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.020165
Epoch 195
Validation binary_cross_entropy = 0.972355
Epoch 196
Validation binary_cross_entropy = 0.945704
Epoch 197
Validation binary_cross_entropy = 0.906285
Epoch 198
Validation binary_cross_entropy = 0.902270
Epoch 199
Loss = 1.1990e-01, PNorm = 62.4412, GNorm = 6.0751, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.927710
Epoch 200
Validation binary_cross_entropy = 0.990344
Epoch 201
Validation binary_cross_entropy = 1.087170
Epoch 202
Validation binary_cross_entropy = 1.136199
Epoch 203
Validation binary_cross_entropy = 1.051365
Epoch 204
Loss = 7.4943e-02, PNorm = 62.5603, GNorm = 5.1913, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.018336
Epoch 205
Validation binary_cross_entropy = 0.999174
Epoch 206
Validation binary_cross_entropy = 0.982431
Epoch 207
Validation binary_cross_entropy = 1.012307
Epoch 208
Validation binary_cross_entropy = 1.042632
Epoch 209
Loss = 5.3579e-02, PNorm = 62.6761, GNorm = 4.0026, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.077661
Epoch 210
Validation binary_cross_entropy = 1.087950
Epoch 211
Validation binary_cross_entropy = 1.130541
Epoch 212
Validation binary_cross_entropy = 1.116406
Epoch 213
Validation binary_cross_entropy = 1.095677
Epoch 214
Loss = 4.7947e-02, PNorm = 62.8105, GNorm = 1.1997, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.078748
Epoch 215
Validation binary_cross_entropy = 1.076599
Epoch 216
Validation binary_cross_entropy = 1.087416
Epoch 217
Validation binary_cross_entropy = 1.098339
Epoch 218
Validation binary_cross_entropy = 1.108291
Epoch 219
Loss = 4.8170e-02, PNorm = 62.9031, GNorm = 3.3118, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.115742
Epoch 220
Validation binary_cross_entropy = 1.108734
Epoch 221
Validation binary_cross_entropy = 1.127118
Epoch 222
Validation binary_cross_entropy = 1.144223
Epoch 223
Validation binary_cross_entropy = 1.170273
Epoch 224
Loss = 1.7984e-02, PNorm = 62.9625, GNorm = 0.4898, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.217600
Epoch 225
Validation binary_cross_entropy = 1.276415
Epoch 226
Validation binary_cross_entropy = 1.189104
Epoch 227
Validation binary_cross_entropy = 1.193989
Epoch 228
Validation binary_cross_entropy = 1.211262
Epoch 229
Loss = 6.9369e-02, PNorm = 63.0264, GNorm = 4.8800, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.068811
Epoch 230
Validation binary_cross_entropy = 1.003498
Epoch 231
Validation binary_cross_entropy = 0.977928
Epoch 232
Validation binary_cross_entropy = 0.964179
Epoch 233
Validation binary_cross_entropy = 0.988404
Epoch 234
Loss = 4.9042e-02, PNorm = 63.1054, GNorm = 2.1888, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.023878
Epoch 235
Validation binary_cross_entropy = 1.058417
Epoch 236
Validation binary_cross_entropy = 1.091313
Epoch 237
Validation binary_cross_entropy = 1.127383
Epoch 238
Validation binary_cross_entropy = 1.142759
Epoch 239
Loss = 1.0604e-02, PNorm = 63.1809, GNorm = 1.7209, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.154888
Epoch 240
Validation binary_cross_entropy = 1.183363
Epoch 241
Validation binary_cross_entropy = 1.189021
Epoch 242
Validation binary_cross_entropy = 1.207574
Epoch 243
Validation binary_cross_entropy = 1.230701
Epoch 244
Loss = 5.2789e-02, PNorm = 63.2861, GNorm = 9.8315, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.260290
Epoch 245
Validation binary_cross_entropy = 1.291939
Epoch 246
Validation binary_cross_entropy = 1.264426
Epoch 247
Validation binary_cross_entropy = 1.220279
Epoch 248
Validation binary_cross_entropy = 1.233994
Epoch 249
Loss = 6.9388e-03, PNorm = 63.3600, GNorm = 0.3819, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.278331
Epoch 250
Validation binary_cross_entropy = 1.315152
Epoch 251
Validation binary_cross_entropy = 1.406667
Epoch 252
Validation binary_cross_entropy = 1.365291
Epoch 253
Validation binary_cross_entropy = 1.260271
Epoch 254
Loss = 4.7089e-02, PNorm = 63.4437, GNorm = 2.0918, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.189112
Epoch 255
Validation binary_cross_entropy = 1.183939
Epoch 256
Validation binary_cross_entropy = 1.131670
Epoch 257
Validation binary_cross_entropy = 1.235311
Epoch 258
Validation binary_cross_entropy = 1.257368
Epoch 259
Loss = 1.0848e-01, PNorm = 63.5769, GNorm = 17.5474, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.162621
Epoch 260
Validation binary_cross_entropy = 1.291245
Epoch 261
Validation binary_cross_entropy = 1.939163
Epoch 262
Validation binary_cross_entropy = 1.217047
Epoch 263
Validation binary_cross_entropy = 1.364704
Epoch 264
Loss = 2.5108e-01, PNorm = 63.7687, GNorm = 21.1576, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.210145
Epoch 265
Validation binary_cross_entropy = 1.070046
Epoch 266
Validation binary_cross_entropy = 1.053865
Epoch 267
Validation binary_cross_entropy = 1.071289
Epoch 268
Validation binary_cross_entropy = 1.109874
Epoch 269
Loss = 1.0584e-01, PNorm = 63.9575, GNorm = 3.9059, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.144053
Epoch 270
Validation binary_cross_entropy = 1.211645
Epoch 271
Validation binary_cross_entropy = 1.304027
Epoch 272
Validation binary_cross_entropy = 1.326096
Epoch 273
Validation binary_cross_entropy = 1.244208
Epoch 274
Loss = 2.0460e-02, PNorm = 64.1438, GNorm = 2.5067, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.197734
Epoch 275
Validation binary_cross_entropy = 1.197827
Epoch 276
Validation binary_cross_entropy = 1.227227
Epoch 277
Validation binary_cross_entropy = 1.253118
Epoch 278
Validation binary_cross_entropy = 1.206116
Epoch 279
Loss = 5.8393e-03, PNorm = 64.2751, GNorm = 0.6959, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.218256
Epoch 280
Validation binary_cross_entropy = 1.256697
Epoch 281
Validation binary_cross_entropy = 1.218533
Epoch 282
Validation binary_cross_entropy = 1.191999
Epoch 283
Validation binary_cross_entropy = 1.191011
Epoch 284
Loss = 8.2297e-03, PNorm = 64.3632, GNorm = 0.1613, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.187899
Epoch 285
Validation binary_cross_entropy = 1.183899
Epoch 286
Validation binary_cross_entropy = 1.180159
Epoch 287
Validation binary_cross_entropy = 1.191805
Epoch 288
Validation binary_cross_entropy = 1.264290
Epoch 289
Loss = 1.3813e-02, PNorm = 64.4115, GNorm = 1.1006, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.314911
Epoch 290
Validation binary_cross_entropy = 1.320728
Epoch 291
Validation binary_cross_entropy = 1.323879
Epoch 292
Validation binary_cross_entropy = 1.313667
Epoch 293
Validation binary_cross_entropy = 1.302941
Epoch 294
Loss = 3.7463e-02, PNorm = 64.4488, GNorm = 0.3749, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.303029
Epoch 295
Validation binary_cross_entropy = 1.304376
Epoch 296
Validation binary_cross_entropy = 1.303087
Epoch 297
Validation binary_cross_entropy = 1.300634
Epoch 298
Validation binary_cross_entropy = 1.298805
Epoch 299
Loss = 1.0096e-02, PNorm = 64.4862, GNorm = 0.9026, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.301881
Model 0 best validation binary_cross_entropy = 0.346249 on epoch 40
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.225867
Ensemble test binary_cross_entropy = 0.225867
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.535518
Epoch 1
Validation binary_cross_entropy = 0.463668
Epoch 2
Validation binary_cross_entropy = 0.742030
Epoch 3
Validation binary_cross_entropy = 0.878675
Epoch 4
Loss = 1.5064e+00, PNorm = 54.1613, GNorm = 19.3981, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.620404
Epoch 5
Validation binary_cross_entropy = 0.467850
Epoch 6
Validation binary_cross_entropy = 0.439389
Epoch 7
Validation binary_cross_entropy = 0.604033
Epoch 8
Validation binary_cross_entropy = 0.820156
Epoch 9
Loss = 1.4044e+00, PNorm = 54.3405, GNorm = 24.0617, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.617121
Epoch 10
Validation binary_cross_entropy = 0.491159
Epoch 11
Validation binary_cross_entropy = 0.474568
Epoch 12
Validation binary_cross_entropy = 0.518295
Epoch 13
Validation binary_cross_entropy = 0.603859
Epoch 14
Loss = 1.2811e+00, PNorm = 54.5215, GNorm = 9.6001, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.703590
Epoch 15
Validation binary_cross_entropy = 0.655765
Epoch 16
Validation binary_cross_entropy = 0.518013
Epoch 17
Validation binary_cross_entropy = 0.383617
Epoch 18
Validation binary_cross_entropy = 0.404121
Epoch 19
Loss = 1.1317e+00, PNorm = 54.6847, GNorm = 28.0646, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.487905
Epoch 20
Validation binary_cross_entropy = 0.433948
Epoch 21
Validation binary_cross_entropy = 0.460879
Epoch 22
Validation binary_cross_entropy = 0.670971
Epoch 23
Validation binary_cross_entropy = 0.348021
Epoch 24
Loss = 1.3251e+00, PNorm = 54.8434, GNorm = 15.3272, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.410512
Epoch 25
Validation binary_cross_entropy = 0.504153
Epoch 26
Validation binary_cross_entropy = 0.380967
Epoch 27
Validation binary_cross_entropy = 0.359778
Epoch 28
Validation binary_cross_entropy = 0.498713
Epoch 29
Loss = 7.6668e-01, PNorm = 54.9816, GNorm = 25.3611, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.717661
Epoch 30
Validation binary_cross_entropy = 0.477957
Epoch 31
Validation binary_cross_entropy = 0.502276
Epoch 32
Validation binary_cross_entropy = 0.723782
Epoch 33
Validation binary_cross_entropy = 0.396764
Epoch 34
Loss = 1.2047e+00, PNorm = 55.1369, GNorm = 28.6793, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.408446
Epoch 35
Validation binary_cross_entropy = 0.779786
Epoch 36
Validation binary_cross_entropy = 0.813922
Epoch 37
Validation binary_cross_entropy = 0.375165
Epoch 38
Validation binary_cross_entropy = 0.364430
Epoch 39
Loss = 7.6897e-01, PNorm = 55.2825, GNorm = 32.6943, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.386717
Epoch 40
Validation binary_cross_entropy = 0.602913
Epoch 41
Validation binary_cross_entropy = 0.659389
Epoch 42
Validation binary_cross_entropy = 0.372813
Epoch 43
Validation binary_cross_entropy = 0.377367
Epoch 44
Loss = 9.3276e-01, PNorm = 55.4510, GNorm = 7.7878, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.377206
Epoch 45
Validation binary_cross_entropy = 0.611931
Epoch 46
Validation binary_cross_entropy = 0.641368
Epoch 47
Validation binary_cross_entropy = 0.392844
Epoch 48
Validation binary_cross_entropy = 0.352779
Epoch 49
Loss = 9.4500e-01, PNorm = 55.6251, GNorm = 40.4713, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.347921
Epoch 50
Validation binary_cross_entropy = 0.452212
Epoch 51
Validation binary_cross_entropy = 0.527493
Epoch 52
Validation binary_cross_entropy = 0.381200
Epoch 53
Validation binary_cross_entropy = 0.361418
Epoch 54
Loss = 8.2262e-01, PNorm = 55.8233, GNorm = 28.0570, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.379760
Epoch 55
Validation binary_cross_entropy = 0.514528
Epoch 56
Validation binary_cross_entropy = 0.398772
Epoch 57
Validation binary_cross_entropy = 0.378202
Epoch 58
Validation binary_cross_entropy = 0.429014
Epoch 59
Loss = 5.1067e-01, PNorm = 56.0252, GNorm = 11.6844, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.460638
Epoch 60
Validation binary_cross_entropy = 0.384086
Epoch 61
Validation binary_cross_entropy = 0.406710
Epoch 62
Validation binary_cross_entropy = 0.469394
Epoch 63
Validation binary_cross_entropy = 0.532434
Epoch 64
Loss = 4.4719e-01, PNorm = 56.2439, GNorm = 12.3147, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.432573
Epoch 65
Validation binary_cross_entropy = 0.444587
Epoch 66
Validation binary_cross_entropy = 0.782366
Epoch 67
Validation binary_cross_entropy = 0.527866
Epoch 68
Validation binary_cross_entropy = 0.461585
Epoch 69
Loss = 4.6163e-01, PNorm = 56.4846, GNorm = 7.0756, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.449144
Epoch 70
Validation binary_cross_entropy = 0.547905
Epoch 71
Validation binary_cross_entropy = 0.455682
Epoch 72
Validation binary_cross_entropy = 0.450160
Epoch 73
Validation binary_cross_entropy = 0.461852
Epoch 74
Loss = 3.1121e-01, PNorm = 56.7562, GNorm = 13.7769, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.479435
Epoch 75
Validation binary_cross_entropy = 0.520377
Epoch 76
Validation binary_cross_entropy = 0.610436
Epoch 77
Validation binary_cross_entropy = 0.608593
Epoch 78
Validation binary_cross_entropy = 1.189142
Epoch 79
Loss = 7.5344e-01, PNorm = 57.0642, GNorm = 31.7697, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.475376
Epoch 80
Validation binary_cross_entropy = 0.437881
Epoch 81
Validation binary_cross_entropy = 0.659334
Epoch 82
Validation binary_cross_entropy = 0.762026
Epoch 83
Validation binary_cross_entropy = 0.458336
Epoch 84
Loss = 3.5617e-01, PNorm = 57.3882, GNorm = 11.4849, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.460710
Epoch 85
Validation binary_cross_entropy = 0.455235
Epoch 86
Validation binary_cross_entropy = 0.537827
Epoch 87
Validation binary_cross_entropy = 0.719819
Epoch 88
Validation binary_cross_entropy = 0.517353
Epoch 89
Loss = 4.0654e-01, PNorm = 57.6975, GNorm = 16.7477, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.522750
Epoch 90
Validation binary_cross_entropy = 0.507668
Epoch 91
Validation binary_cross_entropy = 0.695471
Epoch 92
Validation binary_cross_entropy = 0.563774
Epoch 93
Validation binary_cross_entropy = 0.440032
Epoch 94
Loss = 4.8632e-01, PNorm = 57.9834, GNorm = 20.1475, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.459469
Epoch 95
Validation binary_cross_entropy = 0.471961
Epoch 96
Validation binary_cross_entropy = 0.543393
Epoch 97
Validation binary_cross_entropy = 0.710968
Epoch 98
Validation binary_cross_entropy = 0.729952
Epoch 99
Loss = 3.2573e-01, PNorm = 58.2740, GNorm = 5.8153, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.719704
Epoch 100
Validation binary_cross_entropy = 0.818095
Epoch 101
Validation binary_cross_entropy = 0.578974
Epoch 102
Validation binary_cross_entropy = 0.558431
Epoch 103
Validation binary_cross_entropy = 0.632238
Epoch 104
Loss = 1.9272e-01, PNorm = 58.5568, GNorm = 21.8425, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.644783
Epoch 105
Validation binary_cross_entropy = 0.551563
Epoch 106
Validation binary_cross_entropy = 0.546031
Epoch 107
Validation binary_cross_entropy = 0.520446
Epoch 108
Validation binary_cross_entropy = 0.599368
Epoch 109
Loss = 1.3271e-01, PNorm = 58.8143, GNorm = 4.0828, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.618397
Epoch 110
Validation binary_cross_entropy = 0.574578
Epoch 111
Validation binary_cross_entropy = 0.572872
Epoch 112
Validation binary_cross_entropy = 0.570590
Epoch 113
Validation binary_cross_entropy = 0.580585
Epoch 114
Loss = 2.2000e-01, PNorm = 59.0250, GNorm = 4.3138, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.617958
Epoch 115
Validation binary_cross_entropy = 0.620086
Epoch 116
Validation binary_cross_entropy = 0.603248
Epoch 117
Validation binary_cross_entropy = 0.618284
Epoch 118
Validation binary_cross_entropy = 0.655965
Epoch 119
Loss = 2.5362e-01, PNorm = 59.2168, GNorm = 12.4757, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.730159
Epoch 120
Validation binary_cross_entropy = 0.761984
Epoch 121
Validation binary_cross_entropy = 0.798250
Epoch 122
Validation binary_cross_entropy = 1.271097
Epoch 123
Validation binary_cross_entropy = 0.740860
Epoch 124
Loss = 1.9287e-01, PNorm = 59.4084, GNorm = 2.7717, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.707989
Epoch 125
Validation binary_cross_entropy = 0.656907
Epoch 126
Validation binary_cross_entropy = 0.695424
Epoch 127
Validation binary_cross_entropy = 0.632732
Epoch 128
Validation binary_cross_entropy = 0.643537
Epoch 129
Loss = 1.8349e-01, PNorm = 59.5972, GNorm = 7.9316, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.598331
Epoch 130
Validation binary_cross_entropy = 0.625509
Epoch 131
Validation binary_cross_entropy = 0.629098
Epoch 132
Validation binary_cross_entropy = 0.656926
Epoch 133
Validation binary_cross_entropy = 0.693236
Epoch 134
Loss = 1.1307e-01, PNorm = 59.7899, GNorm = 18.7599, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.720604
Epoch 135
Validation binary_cross_entropy = 0.712019
Epoch 136
Validation binary_cross_entropy = 0.710748
Epoch 137
Validation binary_cross_entropy = 0.716427
Epoch 138
Validation binary_cross_entropy = 0.734704
Epoch 139
Loss = 9.0977e-02, PNorm = 59.9489, GNorm = 6.3257, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.749724
Epoch 140
Validation binary_cross_entropy = 0.767850
Epoch 141
Validation binary_cross_entropy = 0.778611
Epoch 142
Validation binary_cross_entropy = 0.792736
Epoch 143
Validation binary_cross_entropy = 0.778912
Epoch 144
Loss = 1.0153e-01, PNorm = 60.0979, GNorm = 7.8397, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.785182
Epoch 145
Validation binary_cross_entropy = 0.835928
Epoch 146
Validation binary_cross_entropy = 0.953965
Epoch 147
Validation binary_cross_entropy = 0.873803
Epoch 148
Validation binary_cross_entropy = 0.849429
Epoch 149
Loss = 1.8431e-01, PNorm = 60.2406, GNorm = 5.6950, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.821483
Epoch 150
Validation binary_cross_entropy = 0.788235
Epoch 151
Validation binary_cross_entropy = 0.779287
Epoch 152
Validation binary_cross_entropy = 0.768191
Epoch 153
Validation binary_cross_entropy = 0.752114
Epoch 154
Loss = 9.1012e-02, PNorm = 60.3976, GNorm = 3.2462, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.768488
Epoch 155
Validation binary_cross_entropy = 0.822150
Epoch 156
Validation binary_cross_entropy = 0.865023
Epoch 157
Validation binary_cross_entropy = 0.895121
Epoch 158
Validation binary_cross_entropy = 0.888741
Epoch 159
Loss = 3.0834e-02, PNorm = 60.5395, GNorm = 2.0374, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.882730
Epoch 160
Validation binary_cross_entropy = 0.921710
Epoch 161
Validation binary_cross_entropy = 1.125500
Epoch 162
Validation binary_cross_entropy = 0.981974
Epoch 163
Validation binary_cross_entropy = 0.901104
Epoch 164
Loss = 1.6410e-01, PNorm = 60.6943, GNorm = 8.6784, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.937946
Epoch 165
Validation binary_cross_entropy = 0.839835
Epoch 166
Validation binary_cross_entropy = 0.817073
Epoch 167
Validation binary_cross_entropy = 0.751017
Epoch 168
Validation binary_cross_entropy = 0.719349
Epoch 169
Loss = 1.1825e-01, PNorm = 60.8924, GNorm = 6.6409, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.714774
Epoch 170
Validation binary_cross_entropy = 0.717731
Epoch 171
Validation binary_cross_entropy = 0.747021
Epoch 172
Validation binary_cross_entropy = 0.931725
Epoch 173
Validation binary_cross_entropy = 1.033342
Epoch 174
Loss = 1.2311e-01, PNorm = 61.0788, GNorm = 1.4129, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.865154
Epoch 175
Validation binary_cross_entropy = 0.856321
Epoch 176
Validation binary_cross_entropy = 0.766002
Epoch 177
Validation binary_cross_entropy = 0.738003
Epoch 178
Validation binary_cross_entropy = 0.753068
Epoch 179
Loss = 3.1329e-01, PNorm = 61.2593, GNorm = 27.3853, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.737114
Epoch 180
Validation binary_cross_entropy = 0.789326
Epoch 181
Validation binary_cross_entropy = 0.842106
Epoch 182
Validation binary_cross_entropy = 0.862399
Epoch 183
Validation binary_cross_entropy = 0.902883
Epoch 184
Loss = 6.0726e-02, PNorm = 61.4193, GNorm = 6.7051, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.910386
Epoch 185
Validation binary_cross_entropy = 0.899677
Epoch 186
Validation binary_cross_entropy = 0.905189
Epoch 187
Validation binary_cross_entropy = 0.917277
Epoch 188
Validation binary_cross_entropy = 0.917913
Epoch 189
Loss = 4.7692e-02, PNorm = 61.5610, GNorm = 0.7892, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.928956
Epoch 190
Validation binary_cross_entropy = 0.933512
Epoch 191
Validation binary_cross_entropy = 0.921583
Epoch 192
Validation binary_cross_entropy = 0.911523
Epoch 193
Validation binary_cross_entropy = 0.895847
Epoch 194
Loss = 1.0272e-01, PNorm = 61.6525, GNorm = 13.1195, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.910932
Epoch 195
Validation binary_cross_entropy = 0.923350
Epoch 196
Validation binary_cross_entropy = 0.940114
Epoch 197
Validation binary_cross_entropy = 0.961938
Epoch 198
Validation binary_cross_entropy = 0.979587
Epoch 199
Loss = 1.0564e-02, PNorm = 61.7303, GNorm = 1.1284, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.998927
Epoch 200
Validation binary_cross_entropy = 1.022231
Epoch 201
Validation binary_cross_entropy = 1.038780
Epoch 202
Validation binary_cross_entropy = 1.074844
Epoch 203
Validation binary_cross_entropy = 1.111164
Epoch 204
Loss = 1.9806e-02, PNorm = 61.8219, GNorm = 3.1410, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.128613
Epoch 205
Validation binary_cross_entropy = 1.126382
Epoch 206
Validation binary_cross_entropy = 1.158543
Epoch 207
Validation binary_cross_entropy = 1.168379
Epoch 208
Validation binary_cross_entropy = 1.059552
Epoch 209
Loss = 6.8261e-02, PNorm = 61.9437, GNorm = 4.2385, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.160626
Epoch 210
Validation binary_cross_entropy = 1.023225
Epoch 211
Validation binary_cross_entropy = 1.069359
Epoch 212
Validation binary_cross_entropy = 1.217576
Epoch 213
Validation binary_cross_entropy = 0.885598
Epoch 214
Loss = 6.0359e-02, PNorm = 62.1418, GNorm = 3.7203, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.840601
Epoch 215
Validation binary_cross_entropy = 0.848771
Epoch 216
Validation binary_cross_entropy = 0.846884
Epoch 217
Validation binary_cross_entropy = 0.852564
Epoch 218
Validation binary_cross_entropy = 0.898268
Epoch 219
Loss = 8.4363e-03, PNorm = 62.3209, GNorm = 0.5735, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.946990
Epoch 220
Validation binary_cross_entropy = 0.976813
Epoch 221
Validation binary_cross_entropy = 1.008216
Epoch 222
Validation binary_cross_entropy = 1.033581
Epoch 223
Validation binary_cross_entropy = 1.067565
Epoch 224
Loss = 6.9017e-02, PNorm = 62.4487, GNorm = 3.0870, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.109856
Epoch 225
Validation binary_cross_entropy = 1.146691
Epoch 226
Validation binary_cross_entropy = 1.163472
Epoch 227
Validation binary_cross_entropy = 1.159327
Epoch 228
Validation binary_cross_entropy = 1.150546
Epoch 229
Loss = 1.1356e-02, PNorm = 62.5444, GNorm = 2.2381, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.146815
Epoch 230
Validation binary_cross_entropy = 1.125900
Epoch 231
Validation binary_cross_entropy = 1.108463
Epoch 232
Validation binary_cross_entropy = 1.108294
Epoch 233
Validation binary_cross_entropy = 1.138914
Epoch 234
Loss = 1.9969e-02, PNorm = 62.6096, GNorm = 5.4325, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.194193
Epoch 235
Validation binary_cross_entropy = 1.131464
Epoch 236
Validation binary_cross_entropy = 1.108829
Epoch 237
Validation binary_cross_entropy = 1.177296
Epoch 238
Validation binary_cross_entropy = 1.206542
Epoch 239
Loss = 7.7989e-02, PNorm = 62.7219, GNorm = 11.7840, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.136118
Epoch 240
Validation binary_cross_entropy = 1.131164
Epoch 241
Validation binary_cross_entropy = 1.142599
Epoch 242
Validation binary_cross_entropy = 1.129174
Epoch 243
Validation binary_cross_entropy = 1.116072
Epoch 244
Loss = 7.1190e-03, PNorm = 62.8140, GNorm = 0.6121, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.106803
Epoch 245
Validation binary_cross_entropy = 1.113139
Epoch 246
Validation binary_cross_entropy = 1.136798
Epoch 247
Validation binary_cross_entropy = 1.161217
Epoch 248
Validation binary_cross_entropy = 1.167991
Epoch 249
Loss = 6.2903e-02, PNorm = 62.8896, GNorm = 5.7580, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.161538
Epoch 250
Validation binary_cross_entropy = 1.166871
Epoch 251
Validation binary_cross_entropy = 1.175765
Epoch 252
Validation binary_cross_entropy = 1.173280
Epoch 253
Validation binary_cross_entropy = 1.165889
Epoch 254
Loss = 3.6111e-02, PNorm = 62.9535, GNorm = 0.0998, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.159292
Epoch 255
Validation binary_cross_entropy = 1.145863
Epoch 256
Validation binary_cross_entropy = 1.144475
Epoch 257
Validation binary_cross_entropy = 1.156903
Epoch 258
Validation binary_cross_entropy = 1.182605
Epoch 259
Loss = 1.9841e-02, PNorm = 62.9992, GNorm = 2.0253, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.199600
Epoch 260
Validation binary_cross_entropy = 1.222833
Epoch 261
Validation binary_cross_entropy = 1.222897
Epoch 262
Validation binary_cross_entropy = 1.208360
Epoch 263
Validation binary_cross_entropy = 1.219216
Epoch 264
Loss = 9.1749e-02, PNorm = 63.0519, GNorm = 11.3820, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.225958
Epoch 265
Validation binary_cross_entropy = 1.295892
Epoch 266
Validation binary_cross_entropy = 1.306086
Epoch 267
Validation binary_cross_entropy = 1.246745
Epoch 268
Validation binary_cross_entropy = 1.187386
Epoch 269
Loss = 1.9735e-02, PNorm = 63.1366, GNorm = 0.5837, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.237439
Epoch 270
Validation binary_cross_entropy = 1.225369
Epoch 271
Validation binary_cross_entropy = 1.195532
Epoch 272
Validation binary_cross_entropy = 1.210314
Epoch 273
Validation binary_cross_entropy = 1.226782
Epoch 274
Loss = 2.5734e-02, PNorm = 63.2268, GNorm = 3.0871, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.191922
Epoch 275
Validation binary_cross_entropy = 1.157183
Epoch 276
Validation binary_cross_entropy = 1.158152
Epoch 277
Validation binary_cross_entropy = 1.169857
Epoch 278
Validation binary_cross_entropy = 1.193524
Epoch 279
Loss = 6.2836e-02, PNorm = 63.3098, GNorm = 5.0055, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.206299
Epoch 280
Validation binary_cross_entropy = 1.226480
Epoch 281
Validation binary_cross_entropy = 1.278588
Epoch 282
Validation binary_cross_entropy = 1.294211
Epoch 283
Validation binary_cross_entropy = 1.282557
Epoch 284
Loss = 2.0388e-02, PNorm = 63.3851, GNorm = 1.5763, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.286155
Epoch 285
Validation binary_cross_entropy = 1.287796
Epoch 286
Validation binary_cross_entropy = 1.291110
Epoch 287
Validation binary_cross_entropy = 1.314483
Epoch 288
Validation binary_cross_entropy = 1.343547
Epoch 289
Loss = 3.2692e-02, PNorm = 63.4561, GNorm = 3.9571, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.323649
Epoch 290
Validation binary_cross_entropy = 1.309742
Epoch 291
Validation binary_cross_entropy = 1.313369
Epoch 292
Validation binary_cross_entropy = 1.320890
Epoch 293
Validation binary_cross_entropy = 1.362521
Epoch 294
Loss = 4.4337e-02, PNorm = 63.5122, GNorm = 6.5847, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.339323
Epoch 295
Validation binary_cross_entropy = 1.296668
Epoch 296
Validation binary_cross_entropy = 1.313708
Epoch 297
Validation binary_cross_entropy = 1.312657
Epoch 298
Validation binary_cross_entropy = 1.269960
Epoch 299
Loss = 1.5990e-02, PNorm = 63.5843, GNorm = 4.7646, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.249456
Model 0 best validation binary_cross_entropy = 0.347921 on epoch 49
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.201265
Ensemble test binary_cross_entropy = 0.201265
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 3,746,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.546588
Epoch 1
Validation binary_cross_entropy = 0.521550
Epoch 2
Validation binary_cross_entropy = 0.921584
Epoch 3
Validation binary_cross_entropy = 1.241211
Epoch 4
Loss = 1.4156e+00, PNorm = 54.1579, GNorm = 26.7654, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.832143
Epoch 5
Validation binary_cross_entropy = 0.455663
Epoch 6
Validation binary_cross_entropy = 0.392828
Epoch 7
Validation binary_cross_entropy = 0.407025
Epoch 8
Validation binary_cross_entropy = 0.566605
Epoch 9
Loss = 1.4085e+00, PNorm = 54.3225, GNorm = 17.9309, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.092751
Epoch 10
Validation binary_cross_entropy = 0.771284
Epoch 11
Validation binary_cross_entropy = 0.496107
Epoch 12
Validation binary_cross_entropy = 0.468511
Epoch 13
Validation binary_cross_entropy = 0.528054
Epoch 14
Loss = 1.5152e+00, PNorm = 54.4755, GNorm = 15.2883, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.688784
Epoch 15
Validation binary_cross_entropy = 0.796195
Epoch 16
Validation binary_cross_entropy = 0.645233
Epoch 17
Validation binary_cross_entropy = 0.518369
Epoch 18
Validation binary_cross_entropy = 0.456984
Epoch 19
Loss = 1.2410e+00, PNorm = 54.5871, GNorm = 16.5894, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.492603
Epoch 20
Validation binary_cross_entropy = 0.552761
Epoch 21
Validation binary_cross_entropy = 0.612999
Epoch 22
Validation binary_cross_entropy = 0.482948
Epoch 23
Validation binary_cross_entropy = 0.376743
Epoch 24
Loss = 9.7931e-01, PNorm = 54.6939, GNorm = 15.8671, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.410325
Epoch 25
Validation binary_cross_entropy = 0.753778
Epoch 26
Validation binary_cross_entropy = 0.788327
Epoch 27
Validation binary_cross_entropy = 0.402804
Epoch 28
Validation binary_cross_entropy = 0.354192
Epoch 29
Loss = 9.3108e-01, PNorm = 54.8019, GNorm = 35.4771, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.354667
Epoch 30
Validation binary_cross_entropy = 0.401110
Epoch 31
Validation binary_cross_entropy = 0.541480
Epoch 32
Validation binary_cross_entropy = 0.664135
Epoch 33
Validation binary_cross_entropy = 0.510936
Epoch 34
Loss = 9.0216e-01, PNorm = 54.9304, GNorm = 7.7190, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.379207
Epoch 35
Validation binary_cross_entropy = 0.350551
Epoch 36
Validation binary_cross_entropy = 0.400510
Epoch 37
Validation binary_cross_entropy = 0.562887
Epoch 38
Validation binary_cross_entropy = 0.608602
Epoch 39
Loss = 8.1148e-01, PNorm = 55.0801, GNorm = 8.9153, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.351290
Epoch 40
Validation binary_cross_entropy = 0.349746
Epoch 41
Validation binary_cross_entropy = 0.413644
Epoch 42
Validation binary_cross_entropy = 0.530288
Epoch 43
Validation binary_cross_entropy = 0.473723
Epoch 44
Loss = 1.0529e+00, PNorm = 55.1983, GNorm = 17.9177, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.384248
Epoch 45
Validation binary_cross_entropy = 0.363669
Epoch 46
Validation binary_cross_entropy = 0.375059
Epoch 47
Validation binary_cross_entropy = 0.448690
Epoch 48
Validation binary_cross_entropy = 0.584787
Epoch 49
Loss = 8.6515e-01, PNorm = 55.3531, GNorm = 12.7512, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.614984
Epoch 50
Validation binary_cross_entropy = 0.408135
Epoch 51
Validation binary_cross_entropy = 0.347988
Epoch 52
Validation binary_cross_entropy = 0.430875
Epoch 53
Validation binary_cross_entropy = 0.460771
Epoch 54
Loss = 7.5385e-01, PNorm = 55.5431, GNorm = 23.6566, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.350359
Epoch 55
Validation binary_cross_entropy = 0.342395
Epoch 56
Validation binary_cross_entropy = 0.425503
Epoch 57
Validation binary_cross_entropy = 0.482037
Epoch 58
Validation binary_cross_entropy = 0.472122
Epoch 59
Loss = 7.7237e-01, PNorm = 55.7429, GNorm = 24.8691, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.354526
Epoch 60
Validation binary_cross_entropy = 0.352008
Epoch 61
Validation binary_cross_entropy = 0.351075
Epoch 62
Validation binary_cross_entropy = 0.397530
Epoch 63
Validation binary_cross_entropy = 0.433067
Epoch 64
Loss = 3.9030e-01, PNorm = 55.9731, GNorm = 8.6055, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.376506
Epoch 65
Validation binary_cross_entropy = 0.374199
Epoch 66
Validation binary_cross_entropy = 0.474490
Epoch 67
Validation binary_cross_entropy = 0.522839
Epoch 68
Validation binary_cross_entropy = 0.397227
Epoch 69
Loss = 7.3444e-01, PNorm = 56.2189, GNorm = 19.3634, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.391408
Epoch 70
Validation binary_cross_entropy = 0.568760
Epoch 71
Validation binary_cross_entropy = 0.464364
Epoch 72
Validation binary_cross_entropy = 0.381860
Epoch 73
Validation binary_cross_entropy = 0.385287
Epoch 74
Loss = 5.8287e-01, PNorm = 56.4853, GNorm = 7.8779, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.396088
Epoch 75
Validation binary_cross_entropy = 0.449029
Epoch 76
Validation binary_cross_entropy = 0.538279
Epoch 77
Validation binary_cross_entropy = 0.551682
Epoch 78
Validation binary_cross_entropy = 0.492556
Epoch 79
Loss = 6.4188e-01, PNorm = 56.7870, GNorm = 13.3562, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.508031
Epoch 80
Validation binary_cross_entropy = 0.614895
Epoch 81
Validation binary_cross_entropy = 0.445580
Epoch 82
Validation binary_cross_entropy = 0.426779
Epoch 83
Validation binary_cross_entropy = 0.420169
Epoch 84
Loss = 3.6097e-01, PNorm = 57.0742, GNorm = 6.1460, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.445889
Epoch 85
Validation binary_cross_entropy = 0.418290
Epoch 86
Validation binary_cross_entropy = 0.412599
Epoch 87
Validation binary_cross_entropy = 0.426989
Epoch 88
Validation binary_cross_entropy = 0.590260
Epoch 89
Loss = 4.9639e-01, PNorm = 57.3453, GNorm = 13.7407, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.510745
Epoch 90
Validation binary_cross_entropy = 0.482773
Epoch 91
Validation binary_cross_entropy = 0.495512
Epoch 92
Validation binary_cross_entropy = 0.525488
Epoch 93
Validation binary_cross_entropy = 0.468635
Epoch 94
Loss = 2.5389e-01, PNorm = 57.6384, GNorm = 16.1156, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.467296
Epoch 95
Validation binary_cross_entropy = 0.461843
Epoch 96
Validation binary_cross_entropy = 0.475553
Epoch 97
Validation binary_cross_entropy = 0.504894
Epoch 98
Validation binary_cross_entropy = 0.511230
Epoch 99
Loss = 3.7635e-01, PNorm = 57.9466, GNorm = 25.2999, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.550045
Epoch 100
Validation binary_cross_entropy = 0.722342
Epoch 101
Validation binary_cross_entropy = 0.555940
Epoch 102
Validation binary_cross_entropy = 0.526270
Epoch 103
Validation binary_cross_entropy = 0.508558
Epoch 104
Loss = 3.3287e-01, PNorm = 58.2390, GNorm = 13.2578, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.520575
Epoch 105
Validation binary_cross_entropy = 0.538855
Epoch 106
Validation binary_cross_entropy = 0.555384
Epoch 107
Validation binary_cross_entropy = 0.798785
Epoch 108
Validation binary_cross_entropy = 0.624493
Epoch 109
Loss = 2.5584e-01, PNorm = 58.5683, GNorm = 13.4037, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.575102
Epoch 110
Validation binary_cross_entropy = 0.512160
Epoch 111
Validation binary_cross_entropy = 0.506626
Epoch 112
Validation binary_cross_entropy = 0.522902
Epoch 113
Validation binary_cross_entropy = 0.559023
Epoch 114
Loss = 2.1509e-01, PNorm = 58.8603, GNorm = 15.6946, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.661756
Epoch 115
Validation binary_cross_entropy = 0.882916
Epoch 116
Validation binary_cross_entropy = 0.604527
Epoch 117
Validation binary_cross_entropy = 0.558935
Epoch 118
Validation binary_cross_entropy = 0.610658
Epoch 119
Loss = 2.4983e-01, PNorm = 59.1237, GNorm = 21.7127, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.600886
Epoch 120
Validation binary_cross_entropy = 0.578402
Epoch 121
Validation binary_cross_entropy = 0.591647
Epoch 122
Validation binary_cross_entropy = 0.586030
Epoch 123
Validation binary_cross_entropy = 0.623184
Epoch 124
Loss = 1.9578e-01, PNorm = 59.3691, GNorm = 12.7847, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.592553
Epoch 125
Validation binary_cross_entropy = 0.642142
Epoch 126
Validation binary_cross_entropy = 0.552342
Epoch 127
Validation binary_cross_entropy = 0.593924
Epoch 128
Validation binary_cross_entropy = 0.568523
Epoch 129
Loss = 1.3923e-01, PNorm = 59.5863, GNorm = 9.0826, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.640864
Epoch 130
Validation binary_cross_entropy = 0.692293
Epoch 131
Validation binary_cross_entropy = 0.644648
Epoch 132
Validation binary_cross_entropy = 0.617909
Epoch 133
Validation binary_cross_entropy = 0.648221
Epoch 134
Loss = 1.8836e-01, PNorm = 59.7985, GNorm = 18.3751, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.633034
Epoch 135
Validation binary_cross_entropy = 0.633719
Epoch 136
Validation binary_cross_entropy = 0.666866
Epoch 137
Validation binary_cross_entropy = 0.692510
Epoch 138
Validation binary_cross_entropy = 0.797761
Epoch 139
Loss = 1.5876e-01, PNorm = 60.0011, GNorm = 4.8924, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.745839
Epoch 140
Validation binary_cross_entropy = 0.760462
Epoch 141
Validation binary_cross_entropy = 0.751553
Epoch 142
Validation binary_cross_entropy = 0.842393
Epoch 143
Validation binary_cross_entropy = 0.894682
Epoch 144
Loss = 1.0348e-01, PNorm = 60.1906, GNorm = 1.7081, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.791804
Epoch 145
Validation binary_cross_entropy = 0.773407
Epoch 146
Validation binary_cross_entropy = 0.767766
Epoch 147
Validation binary_cross_entropy = 0.729315
Epoch 148
Validation binary_cross_entropy = 0.702987
Epoch 149
Loss = 6.2691e-02, PNorm = 60.3413, GNorm = 2.5669, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.703573
Epoch 150
Validation binary_cross_entropy = 0.705101
Epoch 151
Validation binary_cross_entropy = 0.732275
Epoch 152
Validation binary_cross_entropy = 0.740946
Epoch 153
Validation binary_cross_entropy = 0.767271
Epoch 154
Loss = 2.1011e-01, PNorm = 60.4815, GNorm = 18.3473, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.754641
Epoch 155
Validation binary_cross_entropy = 0.858622
Epoch 156
Validation binary_cross_entropy = 0.756812
Epoch 157
Validation binary_cross_entropy = 0.860730
Epoch 158
Validation binary_cross_entropy = 0.839736
Epoch 159
Loss = 7.3693e-02, PNorm = 60.6559, GNorm = 4.4089, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.795326
Epoch 160
Validation binary_cross_entropy = 0.852970
Epoch 161
Validation binary_cross_entropy = 0.883310
Epoch 162
Validation binary_cross_entropy = 0.946588
Epoch 163
Validation binary_cross_entropy = 0.813344
Epoch 164
Loss = 6.1466e-02, PNorm = 60.8356, GNorm = 3.7085, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.777204
Epoch 165
Validation binary_cross_entropy = 0.802936
Epoch 166
Validation binary_cross_entropy = 0.785645
Epoch 167
Validation binary_cross_entropy = 0.803083
Epoch 168
Validation binary_cross_entropy = 0.742386
Epoch 169
Loss = 1.3850e-01, PNorm = 61.0023, GNorm = 5.4239, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.776940
Epoch 170
Validation binary_cross_entropy = 0.819476
Epoch 171
Validation binary_cross_entropy = 0.832390
Epoch 172
Validation binary_cross_entropy = 0.824628
Epoch 173
Validation binary_cross_entropy = 0.802463
Epoch 174
Loss = 8.6822e-02, PNorm = 61.1561, GNorm = 1.8960, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.798077
Epoch 175
Validation binary_cross_entropy = 0.794735
Epoch 176
Validation binary_cross_entropy = 0.804155
Epoch 177
Validation binary_cross_entropy = 0.838501
Epoch 178
Validation binary_cross_entropy = 0.897615
Epoch 179
Loss = 3.7995e-02, PNorm = 61.2754, GNorm = 2.8623, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.948643
Epoch 180
Validation binary_cross_entropy = 0.983736
Epoch 181
Validation binary_cross_entropy = 1.017951
Epoch 182
Validation binary_cross_entropy = 1.074281
Epoch 183
Validation binary_cross_entropy = 1.102591
Epoch 184
Loss = 3.2803e-02, PNorm = 61.3961, GNorm = 2.8298, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.090997
Epoch 185
Validation binary_cross_entropy = 1.092154
Epoch 186
Validation binary_cross_entropy = 1.092012
Epoch 187
Validation binary_cross_entropy = 1.112794
Epoch 188
Validation binary_cross_entropy = 1.149628
Epoch 189
Loss = 6.5302e-02, PNorm = 61.5134, GNorm = 7.0545, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.072108
Epoch 190
Validation binary_cross_entropy = 0.973666
Epoch 191
Validation binary_cross_entropy = 0.949269
Epoch 192
Validation binary_cross_entropy = 0.969243
Epoch 193
Validation binary_cross_entropy = 0.974512
Epoch 194
Loss = 5.3475e-02, PNorm = 61.6517, GNorm = 4.8372, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.939224
Epoch 195
Validation binary_cross_entropy = 1.108266
Epoch 196
Validation binary_cross_entropy = 1.143603
Epoch 197
Validation binary_cross_entropy = 1.042016
Epoch 198
Validation binary_cross_entropy = 1.016466
Epoch 199
Loss = 7.6796e-02, PNorm = 61.8167, GNorm = 7.5087, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.969116
Epoch 200
Validation binary_cross_entropy = 0.934167
Epoch 201
Validation binary_cross_entropy = 0.963662
Epoch 202
Validation binary_cross_entropy = 1.029764
Epoch 203
Validation binary_cross_entropy = 1.077026
Epoch 204
Loss = 1.6928e-01, PNorm = 61.9388, GNorm = 9.1032, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.065166
Epoch 205
Validation binary_cross_entropy = 1.016132
Epoch 206
Validation binary_cross_entropy = 1.016765
Epoch 207
Validation binary_cross_entropy = 0.996716
Epoch 208
Validation binary_cross_entropy = 0.999144
Epoch 209
Loss = 4.7416e-02, PNorm = 62.0745, GNorm = 10.5784, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.015669
Epoch 210
Validation binary_cross_entropy = 1.095634
Epoch 211
Validation binary_cross_entropy = 1.107112
Epoch 212
Validation binary_cross_entropy = 1.178798
Epoch 213
Validation binary_cross_entropy = 1.129178
Epoch 214
Loss = 2.6912e-02, PNorm = 62.1940, GNorm = 2.2466, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.075297
Epoch 215
Validation binary_cross_entropy = 1.085128
Epoch 216
Validation binary_cross_entropy = 1.082132
Epoch 217
Validation binary_cross_entropy = 1.135931
Epoch 218
Validation binary_cross_entropy = 1.133807
Epoch 219
Loss = 3.2037e-02, PNorm = 62.3072, GNorm = 1.9727, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.130941
Epoch 220
Validation binary_cross_entropy = 1.174963
Epoch 221
Validation binary_cross_entropy = 1.169378
Epoch 222
Validation binary_cross_entropy = 1.159560
Epoch 223
Validation binary_cross_entropy = 1.143594
Epoch 224
Loss = 6.7589e-02, PNorm = 62.4092, GNorm = 0.1409, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.132688
Epoch 225
Validation binary_cross_entropy = 1.130438
Epoch 226
Validation binary_cross_entropy = 1.138893
Epoch 227
Validation binary_cross_entropy = 1.158816
Epoch 228
Validation binary_cross_entropy = 1.083148
Epoch 229
Loss = 2.7143e-02, PNorm = 62.4895, GNorm = 3.0927, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.087247
Epoch 230
Validation binary_cross_entropy = 1.139144
Epoch 231
Validation binary_cross_entropy = 1.187744
Epoch 232
Validation binary_cross_entropy = 1.146002
Epoch 233
Validation binary_cross_entropy = 1.141221
Epoch 234
Loss = 7.8615e-02, PNorm = 62.5631, GNorm = 8.3947, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.108005
Epoch 235
Validation binary_cross_entropy = 1.063064
Epoch 236
Validation binary_cross_entropy = 1.046420
Epoch 237
Validation binary_cross_entropy = 1.055211
Epoch 238
Validation binary_cross_entropy = 1.040129
Epoch 239
Loss = 3.6176e-02, PNorm = 62.6424, GNorm = 3.9963, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.010341
Epoch 240
Validation binary_cross_entropy = 0.989784
Epoch 241
Validation binary_cross_entropy = 1.033644
Epoch 242
Validation binary_cross_entropy = 1.062240
Epoch 243
Validation binary_cross_entropy = 1.072450
Epoch 244
Loss = 2.4361e-02, PNorm = 62.7267, GNorm = 1.6972, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.089186
Epoch 245
Validation binary_cross_entropy = 1.152318
Epoch 246
Validation binary_cross_entropy = 1.216020
Epoch 247
Validation binary_cross_entropy = 1.254107
Epoch 248
Validation binary_cross_entropy = 1.248211
Epoch 249
Loss = 5.2341e-02, PNorm = 62.7966, GNorm = 1.4833, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.302493
Epoch 250
Validation binary_cross_entropy = 1.328597
Epoch 251
Validation binary_cross_entropy = 1.138430
Epoch 252
Validation binary_cross_entropy = 1.060264
Epoch 253
Validation binary_cross_entropy = 1.039664
Epoch 254
Loss = 9.8150e-02, PNorm = 62.8801, GNorm = 5.9346, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.091265
Epoch 255
Validation binary_cross_entropy = 1.129959
Epoch 256
Validation binary_cross_entropy = 1.137406
Epoch 257
Validation binary_cross_entropy = 1.211168
Epoch 258
Validation binary_cross_entropy = 1.247431
Epoch 259
Loss = 2.0425e-02, PNorm = 62.9983, GNorm = 0.1293, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.287671
Epoch 260
Validation binary_cross_entropy = 1.340542
Epoch 261
Validation binary_cross_entropy = 1.376018
Epoch 262
Validation binary_cross_entropy = 1.387584
Epoch 263
Validation binary_cross_entropy = 1.402070
Epoch 264
Loss = 2.7909e-02, PNorm = 63.0945, GNorm = 1.0872, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.470461
Epoch 265
Validation binary_cross_entropy = 1.525508
Epoch 266
Validation binary_cross_entropy = 1.563644
Epoch 267
Validation binary_cross_entropy = 1.536032
Epoch 268
Validation binary_cross_entropy = 1.514060
Epoch 269
Loss = 1.0383e-01, PNorm = 63.1592, GNorm = 11.4490, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.571905
Epoch 270
Validation binary_cross_entropy = 1.585870
Epoch 271
Validation binary_cross_entropy = 1.497935
Epoch 272
Validation binary_cross_entropy = 1.426796
Epoch 273
Validation binary_cross_entropy = 1.387283
Epoch 274
Loss = 4.8806e-02, PNorm = 63.2502, GNorm = 12.0765, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.420892
Epoch 275
Validation binary_cross_entropy = 1.320119
Epoch 276
Validation binary_cross_entropy = 1.286367
Epoch 277
Validation binary_cross_entropy = 1.227884
Epoch 278
Validation binary_cross_entropy = 1.192704
Epoch 279
Loss = 2.5140e-02, PNorm = 63.3618, GNorm = 3.8916, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.229855
Epoch 280
Validation binary_cross_entropy = 1.320155
Epoch 281
Validation binary_cross_entropy = 1.409921
Epoch 282
Validation binary_cross_entropy = 1.499338
Epoch 283
Validation binary_cross_entropy = 1.580646
Epoch 284
Loss = 2.4226e-02, PNorm = 63.4549, GNorm = 1.2107, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.482521
Epoch 285
Validation binary_cross_entropy = 1.418112
Epoch 286
Validation binary_cross_entropy = 1.347909
Epoch 287
Validation binary_cross_entropy = 1.302857
Epoch 288
Validation binary_cross_entropy = 1.290855
Epoch 289
Loss = 1.1421e-02, PNorm = 63.5746, GNorm = 1.6879, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.254605
Epoch 290
Validation binary_cross_entropy = 1.249924
Epoch 291
Validation binary_cross_entropy = 1.251729
Epoch 292
Validation binary_cross_entropy = 1.274613
Epoch 293
Validation binary_cross_entropy = 1.340632
Epoch 294
Loss = 1.5902e-02, PNorm = 63.6429, GNorm = 2.4737, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.385806
Epoch 295
Validation binary_cross_entropy = 1.379469
Epoch 296
Validation binary_cross_entropy = 1.366324
Epoch 297
Validation binary_cross_entropy = 1.373728
Epoch 298
Validation binary_cross_entropy = 1.391279
Epoch 299
Loss = 2.3645e-02, PNorm = 63.6883, GNorm = 1.3769, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.407801
Model 0 best validation binary_cross_entropy = 0.342395 on epoch 55
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.238621
Ensemble test binary_cross_entropy = 0.238621
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.209651
	Seed 1 ==> test binary_cross_entropy = 0.231419
	Seed 2 ==> test binary_cross_entropy = 0.263363
	Seed 3 ==> test binary_cross_entropy = 0.250060
	Seed 4 ==> test binary_cross_entropy = 0.189909
	Seed 5 ==> test binary_cross_entropy = 0.203085
	Seed 6 ==> test binary_cross_entropy = 0.255989
	Seed 7 ==> test binary_cross_entropy = 0.225867
	Seed 8 ==> test binary_cross_entropy = 0.201265
	Seed 9 ==> test binary_cross_entropy = 0.238621
Overall test binary_cross_entropy = 0.226923 +/- 0.024002
Elapsed time = 0:08:25
